agora inbox for [email protected]help / color / mirror / Atom feed
[PATCH 4/4] Add CONCURRENTLY option to REPACK command. 276+ messages / 2 participants [nested] [flat]
* [PATCH 4/4] Add CONCURRENTLY option to REPACK command. @ 2025-08-11 14:12 Antonin Houska <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Antonin Houska @ 2025-08-11 14:12 UTC (permalink / raw) The REPACK command copies the relation data into a new file, creates new indexes and eventually swaps the files. To make sure that the old file does not change during the copying, the relation is locked in an exclusive mode, which prevents applications from both reading and writing. (To keep the data consistent, we'd only need to prevent the applications from writing, but even reading needs to be blocked before we can swap the files - otherwise some applications could continue using the old file. Since we should not request a stronger lock without releasing the weaker one first, we acquire the exclusive lock in the beginning and keep it till the end of the processing.) This patch introduces an alternative workflow, which only requires the exclusive lock when the relation (and index) files are being swapped. (Supposedly, the swapping should be pretty fast.) On the other hand, when we copy the data to the new file, we allow applications to read from the relation and even to write to it. First, we scan the relation using a "historic snapshot", and insert all the tuples satisfying this snapshot into the new file. Second, logical decoding is used to capture the data changes done by applications during the copying (i.e. changes that do not satisfy the historic snapshot mentioned above), and those are applied to the new file before we acquire the exclusive lock that we need to swap the files. (Of course, more data changes can take place while we are waiting for the lock - these will be applied to the new file after we have acquired the lock, before we swap the files.) Since the logical decoding system, during its startup, waits until all the transactions which already have XID assigned have finished, there is a risk of deadlock if a transaction that already changed anything in the database tries to acquire a conflicting lock on the table REPACK CONCURRENTLY is working on. As an example, consider transaction running CREATE INDEX command on the table that is being REPACKed CONCURRENTLY. On the other hand, DML commands (INSERT, UPDATE, DELETE) are not a problem as their lock does not conflict with REPACK CONCURRENTLY. The current approach is that we accept the risk. If we tried to avoid it, it'd be necessary to unlock the table before the logical decoding is setup and lock it again afterwards. Such temporary unlocking would imply re-checking if the table still meets all the requirements for REPACK CONCURRENTLY. Like the existing implementation of REPACK, the variant with the CONCURRENTLY option also requires an extra space for the new relation and index files (which coexist with the old files for some time). In addition, the CONCURRENTLY option might introduce a lag in releasing WAL segments for archiving / recycling. This is due to the decoding of the data changes done by applications concurrently. When copying the table contents into the new file, we check the lag periodically. If it exceeds the size of a WAL segment, we decode all the available WAL before resuming the copying. (Of course, the changes are not applied until the whole table contents is copied.) A background worker might be a better approach for the decoding - let's consider implementing it in the future. The WAL records produced by running DML commands on the new relation do not contain enough information to be processed by the logical decoding system. All we need from the new relation is the file (relfilenode), while the actual relation is eventually dropped. Thus there is no point in replaying the DMLs anywhere. --- doc/src/sgml/monitoring.sgml | 37 +- doc/src/sgml/mvcc.sgml | 12 +- doc/src/sgml/ref/repack.sgml | 129 +- src/Makefile | 1 + src/backend/access/heap/heapam.c | 34 +- src/backend/access/heap/heapam_handler.c | 215 ++- src/backend/access/heap/rewriteheap.c | 6 +- src/backend/access/transam/xact.c | 11 +- src/backend/catalog/system_views.sql | 30 +- src/backend/commands/cluster.c | 1677 +++++++++++++++-- src/backend/commands/matview.c | 2 +- src/backend/commands/tablecmds.c | 1 + src/backend/commands/vacuum.c | 12 +- src/backend/meson.build | 1 + src/backend/replication/logical/decode.c | 83 + src/backend/replication/logical/snapbuild.c | 20 + .../replication/pgoutput_repack/Makefile | 32 + .../replication/pgoutput_repack/meson.build | 18 + .../pgoutput_repack/pgoutput_repack.c | 288 +++ src/backend/storage/ipc/ipci.c | 1 + .../storage/lmgr/generate-lwlocknames.pl | 2 +- src/backend/utils/cache/relcache.c | 1 + src/backend/utils/time/snapmgr.c | 3 +- src/bin/psql/tab-complete.in.c | 25 +- src/include/access/heapam.h | 9 +- src/include/access/heapam_xlog.h | 2 + src/include/access/tableam.h | 10 + src/include/commands/cluster.h | 90 +- src/include/commands/progress.h | 23 +- src/include/replication/snapbuild.h | 1 + src/include/storage/lockdefs.h | 4 +- src/include/utils/snapmgr.h | 2 + src/test/modules/injection_points/Makefile | 5 +- .../injection_points/expected/repack.out | 113 ++ .../modules/injection_points/logical.conf | 1 + src/test/modules/injection_points/meson.build | 4 + .../injection_points/specs/repack.spec | 143 ++ src/test/regress/expected/rules.out | 29 +- src/tools/pgindent/typedefs.list | 4 + 39 files changed, 2820 insertions(+), 261 deletions(-) create mode 100644 src/backend/replication/pgoutput_repack/Makefile create mode 100644 src/backend/replication/pgoutput_repack/meson.build create mode 100644 src/backend/replication/pgoutput_repack/pgoutput_repack.c create mode 100644 src/test/modules/injection_points/expected/repack.out create mode 100644 src/test/modules/injection_points/logical.conf create mode 100644 src/test/modules/injection_points/specs/repack.spec diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 12e103d319d..61c0197555f 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -6074,14 +6074,35 @@ FROM pg_stat_get_backend_idset() AS backendid; <row> <entry role="catalog_table_entry"><para role="column_definition"> - <structfield>heap_tuples_written</structfield> <type>bigint</type> + <structfield>heap_tuples_inserted</structfield> <type>bigint</type> </para> <para> - Number of heap tuples written. + Number of heap tuples inserted. This counter only advances when the phase is <literal>seq scanning heap</literal>, - <literal>index scanning heap</literal> - or <literal>writing new heap</literal>. + <literal>index scanning heap</literal>, + <literal>writing new heap</literal> + or <literal>catch-up</literal>. + </para></entry> + </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>heap_tuples_updated</structfield> <type>bigint</type> + </para> + <para> + Number of heap tuples updated. + This counter only advances when the phase is <literal>catch-up</literal>. + </para></entry> + </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>heap_tuples_deleted</structfield> <type>bigint</type> + </para> + <para> + Number of heap tuples deleted. + This counter only advances when the phase is <literal>catch-up</literal>. </para></entry> </row> @@ -6162,6 +6183,14 @@ FROM pg_stat_get_backend_idset() AS backendid; <command>REPACK</command> is currently writing the new heap. </entry> </row> + <row> + <entry><literal>catch-up</literal></entry> + <entry> + <command>REPACK CONCURRENTLY</command> is currently processing the DML + commands that other transactions executed during any of the preceding + phase. + </entry> + </row> <row> <entry><literal>swapping relation files</literal></entry> <entry> diff --git a/doc/src/sgml/mvcc.sgml b/doc/src/sgml/mvcc.sgml index 049ee75a4ba..0f5c34af542 100644 --- a/doc/src/sgml/mvcc.sgml +++ b/doc/src/sgml/mvcc.sgml @@ -1833,15 +1833,17 @@ SELECT pg_advisory_lock(q.id) FROM <title>Caveats</title> <para> - Some DDL commands, currently only <link linkend="sql-truncate"><command>TRUNCATE</command></link> and the - table-rewriting forms of <link linkend="sql-altertable"><command>ALTER TABLE</command></link>, are not + Some commands, currently only <link linkend="sql-truncate"><command>TRUNCATE</command></link>, the + table-rewriting forms of <link linkend="sql-altertable"><command>ALTER + TABLE</command></link> and <command>REPACK</command> with + the <literal>CONCURRENTLY</literal> option, are not MVCC-safe. This means that after the truncation or rewrite commits, the table will appear empty to concurrent transactions, if they are using a - snapshot taken before the DDL command committed. This will only be an + snapshot taken before the command committed. This will only be an issue for a transaction that did not access the table in question - before the DDL command started — any transaction that has done so + before the command started — any transaction that has done so would hold at least an <literal>ACCESS SHARE</literal> table lock, - which would block the DDL command until that transaction completes. + which would block the truncating or rewriting command until that transaction completes. So these commands will not cause any apparent inconsistency in the table contents for successive queries on the target table, but they could cause visible inconsistency between the contents of the target diff --git a/doc/src/sgml/ref/repack.sgml b/doc/src/sgml/ref/repack.sgml index a612c72d971..9c089a6b3d7 100644 --- a/doc/src/sgml/ref/repack.sgml +++ b/doc/src/sgml/ref/repack.sgml @@ -22,6 +22,7 @@ PostgreSQL documentation <refsynopsisdiv> <synopsis> REPACK [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] [ <replaceable class="parameter">table_name</replaceable> [ USING INDEX <replaceable class="parameter">index_name</replaceable> ] ] +REPACK [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] CONCURRENTLY <replaceable class="parameter">table_name</replaceable> [ USING INDEX <replaceable class="parameter">index_name</replaceable> ] <phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase> @@ -48,7 +49,8 @@ REPACK [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] [ <re processes every table and materialized view in the current database that the current user has the <literal>MAINTAIN</literal> privilege on. This form of <command>REPACK</command> cannot be executed inside a transaction - block. + block. Also, this form is not allowed if + the <literal>CONCURRENTLY</literal> option is used. </para> <para> @@ -61,7 +63,8 @@ REPACK [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] [ <re When a table is being repacked, an <literal>ACCESS EXCLUSIVE</literal> lock is acquired on it. This prevents any other database operations (both reads and writes) from operating on the table until the <command>REPACK</command> - is finished. + is finished. If you want to keep the table accessible during the repacking, + consider using the <literal>CONCURRENTLY</literal> option. </para> <refsect2 id="sql-repack-notes-on-clustering" xreflabel="Notes on Clustering"> @@ -160,6 +163,128 @@ REPACK [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] [ <re </listitem> </varlistentry> + <varlistentry> + <term><literal>CONCURRENTLY</literal></term> + <listitem> + <para> + Allow other transactions to use the table while it is being repacked. + </para> + + <para> + Internally, <command>REPACK</command> copies the contents of the table + (ignoring dead tuples) into a new file, sorted by the specified index, + and also creates a new file for each index. Then it swaps the old and + new files for the table and all the indexes, and deletes the old + files. The <literal>ACCESS EXCLUSIVE</literal> lock is needed to make + sure that the old files do not change during the processing because the + changes would get lost due to the swap. + </para> + + <para> + With the <literal>CONCURRENTLY</literal> option, the <literal>ACCESS + EXCLUSIVE</literal> lock is only acquired to swap the table and index + files. The data changes that took place during the creation of the new + table and index files are captured using logical decoding + (<xref linkend="logicaldecoding"/>) and applied before + the <literal>ACCESS EXCLUSIVE</literal> lock is requested. Thus the lock + is typically held only for the time needed to swap the files, which + should be pretty short. However, the time might still be noticeable if + too many data changes have been done to the table while + <command>REPACK</command> was waiting for the lock: those changes must + be processed just before the files are swapped, while the + <literal>ACCESS EXCLUSIVE</literal> lock is being held. + </para> + + <para> + Note that <command>REPACK</command> with the + the <literal>CONCURRENTLY</literal> option does not try to order the + rows inserted into the table after the repacking started. Also + note <command>REPACK</command> might fail to complete due to DDL + commands executed on the table by other transactions during the + repacking. + </para> + + <note> + <para> + In addition to the temporary space requirements explained in + <xref linkend="sql-repack-notes-on-resources"/>, + the <literal>CONCURRENTLY</literal> option can add to the usage of + temporary space a bit more. The reason is that other transactions can + perform DML operations which cannot be applied to the new file until + <command>REPACK</command> has copied all the tuples from the old + file. Thus the tuples inserted into the old file during the copying are + also stored separately in a temporary file, so they can eventually be + applied to the new file. + </para> + + <para> + Furthermore, the data changes performed during the copying are + extracted from <link linkend="wal">write-ahead log</link> (WAL), and + this extraction (decoding) only takes place when certain amount of WAL + has been written. Therefore, WAL removal can be delayed by this + threshold. Currently the threshold is equal to the value of + the <link linkend="guc-wal-segment-size"><varname>wal_segment_size</varname></link> + configuration parameter. + </para> + </note> + + <para> + The <literal>CONCURRENTLY</literal> option cannot be used in the + following cases: + + <itemizedlist> + <listitem> + <para> + The table is <literal>UNLOGGED</literal>. + </para> + </listitem> + + <listitem> + <para> + The table is partitioned. + </para> + </listitem> + + <listitem> + <para> + The table is a system catalog or a <acronym>TOAST</acronym> table. + </para> + </listitem> + + <listitem> + <para> + <command>REPACK</command> is executed inside a transaction block. + </para> + </listitem> + + <listitem> + <para> + The <link linkend="guc-wal-level"><varname>wal_level</varname></link> + configuration parameter is less than <literal>logical</literal>. + </para> + </listitem> + + <listitem> + <para> + The <link linkend="guc-max-replication-slots"><varname>max_replication_slots</varname></link> + configuration parameter does not allow for creation of an additional + replication slot. + </para> + </listitem> + </itemizedlist> + </para> + + <warning> + <para> + <command>REPACK</command> with the <literal>CONCURRENTLY</literal> + option is not MVCC-safe, see <xref linkend="mvcc-caveats"/> for + details. + </para> + </warning> + + </listitem> + </varlistentry> + <varlistentry> <term><literal>VERBOSE</literal></term> <listitem> diff --git a/src/Makefile b/src/Makefile index 2f31a2f20a7..b18c9a14ffa 100644 --- a/src/Makefile +++ b/src/Makefile @@ -23,6 +23,7 @@ SUBDIRS = \ interfaces \ backend/replication/libpqwalreceiver \ backend/replication/pgoutput \ + backend/replication/pgoutput_repack \ fe_utils \ bin \ pl \ diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 0dcd6ee817e..4fdb3e880e4 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -60,7 +60,8 @@ static HeapTuple heap_prepare_insert(Relation relation, HeapTuple tup, static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf, Buffer newbuf, HeapTuple oldtup, HeapTuple newtup, HeapTuple old_key_tuple, - bool all_visible_cleared, bool new_all_visible_cleared); + bool all_visible_cleared, bool new_all_visible_cleared, + bool wal_logical); #ifdef USE_ASSERT_CHECKING static void check_lock_if_inplace_updateable_rel(Relation relation, ItemPointer otid, @@ -2769,7 +2770,7 @@ xmax_infomask_changed(uint16 new_infomask, uint16 old_infomask) TM_Result heap_delete(Relation relation, ItemPointer tid, CommandId cid, Snapshot crosscheck, bool wait, - TM_FailureData *tmfd, bool changingPart) + TM_FailureData *tmfd, bool changingPart, bool wal_logical) { TM_Result result; TransactionId xid = GetCurrentTransactionId(); @@ -3016,7 +3017,8 @@ l1: * Compute replica identity tuple before entering the critical section so * we don't PANIC upon a memory allocation failure. */ - old_key_tuple = ExtractReplicaIdentity(relation, &tp, true, &old_key_copied); + old_key_tuple = wal_logical ? + ExtractReplicaIdentity(relation, &tp, true, &old_key_copied) : NULL; /* * If this is the first possibly-multixact-able operation in the current @@ -3106,6 +3108,15 @@ l1: xlrec.flags |= XLH_DELETE_CONTAINS_OLD_KEY; } + /* + * Unlike UPDATE, DELETE is decoded even if there is no old key, so it + * does not help to clear both XLH_DELETE_CONTAINS_OLD_TUPLE and + * XLH_DELETE_CONTAINS_OLD_KEY. Thus we need an extra flag. TODO + * Consider not decoding tuples w/o the old tuple/key instead. + */ + if (!wal_logical) + xlrec.flags |= XLH_DELETE_NO_LOGICAL; + XLogBeginInsert(); XLogRegisterData(&xlrec, SizeOfHeapDelete); @@ -3198,7 +3209,8 @@ simple_heap_delete(Relation relation, ItemPointer tid) result = heap_delete(relation, tid, GetCurrentCommandId(true), InvalidSnapshot, true /* wait for commit */ , - &tmfd, false /* changingPart */ ); + &tmfd, false, /* changingPart */ + true /* wal_logical */); switch (result) { case TM_SelfModified: @@ -3239,7 +3251,7 @@ TM_Result heap_update(Relation relation, ItemPointer otid, HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode, - TU_UpdateIndexes *update_indexes) + TU_UpdateIndexes *update_indexes, bool wal_logical) { TM_Result result; TransactionId xid = GetCurrentTransactionId(); @@ -4132,7 +4144,8 @@ l2: newbuf, &oldtup, heaptup, old_key_tuple, all_visible_cleared, - all_visible_cleared_new); + all_visible_cleared_new, + wal_logical); if (newbuf != buffer) { PageSetLSN(BufferGetPage(newbuf), recptr); @@ -4490,7 +4503,8 @@ simple_heap_update(Relation relation, ItemPointer otid, HeapTuple tup, result = heap_update(relation, otid, tup, GetCurrentCommandId(true), InvalidSnapshot, true /* wait for commit */ , - &tmfd, &lockmode, update_indexes); + &tmfd, &lockmode, update_indexes, + true /* wal_logical */); switch (result) { case TM_SelfModified: @@ -8831,7 +8845,8 @@ static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf, Buffer newbuf, HeapTuple oldtup, HeapTuple newtup, HeapTuple old_key_tuple, - bool all_visible_cleared, bool new_all_visible_cleared) + bool all_visible_cleared, bool new_all_visible_cleared, + bool wal_logical) { xl_heap_update xlrec; xl_heap_header xlhdr; @@ -8842,7 +8857,8 @@ log_heap_update(Relation reln, Buffer oldbuf, suffixlen = 0; XLogRecPtr recptr; Page page = BufferGetPage(newbuf); - bool need_tuple_data = RelationIsLogicallyLogged(reln); + bool need_tuple_data = RelationIsLogicallyLogged(reln) && + wal_logical; bool init; int bufflags; diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 0b03070d394..c829c06f769 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -33,6 +33,7 @@ #include "catalog/index.h" #include "catalog/storage.h" #include "catalog/storage_xlog.h" +#include "commands/cluster.h" #include "commands/progress.h" #include "executor/executor.h" #include "miscadmin.h" @@ -309,7 +310,8 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid, * the storage itself is cleaning the dead tuples by itself, it is the * time to call the index tuple deletion also. */ - return heap_delete(relation, tid, cid, crosscheck, wait, tmfd, changingPart); + return heap_delete(relation, tid, cid, crosscheck, wait, tmfd, changingPart, + true); } @@ -328,7 +330,7 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot, tuple->t_tableOid = slot->tts_tableOid; result = heap_update(relation, otid, tuple, cid, crosscheck, wait, - tmfd, lockmode, update_indexes); + tmfd, lockmode, update_indexes, true); ItemPointerCopy(&tuple->t_self, &slot->tts_tid); /* @@ -685,13 +687,15 @@ static void heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, Relation OldIndex, bool use_sort, TransactionId OldestXmin, + Snapshot snapshot, + LogicalDecodingContext *decoding_ctx, TransactionId *xid_cutoff, MultiXactId *multi_cutoff, double *num_tuples, double *tups_vacuumed, double *tups_recently_dead) { - RewriteState rwstate; + RewriteState rwstate = NULL; IndexScanDesc indexScan; TableScanDesc tableScan; HeapScanDesc heapScan; @@ -705,6 +709,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, bool *isnull; BufferHeapTupleTableSlot *hslot; BlockNumber prev_cblock = InvalidBlockNumber; + bool concurrent = snapshot != NULL; + XLogRecPtr end_of_wal_prev = GetFlushRecPtr(NULL); /* Remember if it's a system catalog */ is_system_catalog = IsSystemRelation(OldHeap); @@ -720,9 +726,12 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, values = (Datum *) palloc(natts * sizeof(Datum)); isnull = (bool *) palloc(natts * sizeof(bool)); - /* Initialize the rewrite operation */ - rwstate = begin_heap_rewrite(OldHeap, NewHeap, OldestXmin, *xid_cutoff, - *multi_cutoff); + /* + * Initialize the rewrite operation. + */ + if (!concurrent) + rwstate = begin_heap_rewrite(OldHeap, NewHeap, OldestXmin, + *xid_cutoff, *multi_cutoff); /* Set up sorting if wanted */ @@ -737,6 +746,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, * Prepare to scan the OldHeap. To ensure we see recently-dead tuples * that still need to be copied, we scan with SnapshotAny and use * HeapTupleSatisfiesVacuum for the visibility test. + * + * In the CONCURRENTLY case, we do regular MVCC visibility tests, using + * the snapshot passed by the caller. */ if (OldIndex != NULL && !use_sort) { @@ -753,7 +765,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, tableScan = NULL; heapScan = NULL; - indexScan = index_beginscan(OldHeap, OldIndex, SnapshotAny, NULL, 0, 0); + indexScan = index_beginscan(OldHeap, OldIndex, + snapshot ? snapshot :SnapshotAny, + NULL, 0, 0); index_rescan(indexScan, NULL, 0, NULL, 0); } else @@ -762,7 +776,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, pgstat_progress_update_param(PROGRESS_REPACK_PHASE, PROGRESS_REPACK_PHASE_SEQ_SCAN_HEAP); - tableScan = table_beginscan(OldHeap, SnapshotAny, 0, (ScanKey) NULL); + tableScan = table_beginscan(OldHeap, + snapshot ? snapshot :SnapshotAny, + 0, (ScanKey) NULL); heapScan = (HeapScanDesc) tableScan; indexScan = NULL; @@ -785,6 +801,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, HeapTuple tuple; Buffer buf; bool isdead; + HTSV_Result vis; CHECK_FOR_INTERRUPTS(); @@ -837,70 +854,84 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, tuple = ExecFetchSlotHeapTuple(slot, false, NULL); buf = hslot->buffer; - LockBuffer(buf, BUFFER_LOCK_SHARE); - - switch (HeapTupleSatisfiesVacuum(tuple, OldestXmin, buf)) + /* + * Regarding CONCURRENTLY, see the comments on MVCC snapshot above. + */ + if (!concurrent) { - case HEAPTUPLE_DEAD: - /* Definitely dead */ - isdead = true; - break; - case HEAPTUPLE_RECENTLY_DEAD: - *tups_recently_dead += 1; - /* fall through */ - case HEAPTUPLE_LIVE: - /* Live or recently dead, must copy it */ - isdead = false; - break; - case HEAPTUPLE_INSERT_IN_PROGRESS: + LockBuffer(buf, BUFFER_LOCK_SHARE); - /* - * Since we hold exclusive lock on the relation, normally the - * only way to see this is if it was inserted earlier in our - * own transaction. However, it can happen in system - * catalogs, since we tend to release write lock before commit - * there. Give a warning if neither case applies; but in any - * case we had better copy it. - */ - if (!is_system_catalog && - !TransactionIdIsCurrentTransactionId(HeapTupleHeaderGetXmin(tuple->t_data))) - elog(WARNING, "concurrent insert in progress within table \"%s\"", - RelationGetRelationName(OldHeap)); - /* treat as live */ - isdead = false; - break; - case HEAPTUPLE_DELETE_IN_PROGRESS: + switch ((vis = HeapTupleSatisfiesVacuum(tuple, OldestXmin, buf))) + { + case HEAPTUPLE_DEAD: + /* Definitely dead */ + isdead = true; + break; + case HEAPTUPLE_RECENTLY_DEAD: + *tups_recently_dead += 1; + /* fall through */ + case HEAPTUPLE_LIVE: + /* Live or recently dead, must copy it */ + isdead = false; + break; + case HEAPTUPLE_INSERT_IN_PROGRESS: /* - * Similar situation to INSERT_IN_PROGRESS case. + * As long as we hold exclusive lock on the relation, normally + * the only way to see this is if it was inserted earlier in + * our own transaction. However, it can happen in system + * catalogs, since we tend to release write lock before commit + * there. Also, there's no exclusive lock during concurrent + * processing. Give a warning if neither case applies; but in + * any case we had better copy it. */ - if (!is_system_catalog && - !TransactionIdIsCurrentTransactionId(HeapTupleHeaderGetUpdateXid(tuple->t_data))) - elog(WARNING, "concurrent delete in progress within table \"%s\"", - RelationGetRelationName(OldHeap)); - /* treat as recently dead */ - *tups_recently_dead += 1; - isdead = false; - break; - default: - elog(ERROR, "unexpected HeapTupleSatisfiesVacuum result"); - isdead = false; /* keep compiler quiet */ - break; - } + if (!is_system_catalog && !concurrent && + !TransactionIdIsCurrentTransactionId(HeapTupleHeaderGetXmin(tuple->t_data))) + elog(WARNING, "concurrent insert in progress within table \"%s\"", + RelationGetRelationName(OldHeap)); + /* treat as live */ + isdead = false; + break; + case HEAPTUPLE_DELETE_IN_PROGRESS: - LockBuffer(buf, BUFFER_LOCK_UNLOCK); + /* + * Similar situation to INSERT_IN_PROGRESS case. + */ + if (!is_system_catalog && !concurrent && + !TransactionIdIsCurrentTransactionId(HeapTupleHeaderGetUpdateXid(tuple->t_data))) + elog(WARNING, "concurrent delete in progress within table \"%s\"", + RelationGetRelationName(OldHeap)); + /* treat as recently dead */ + *tups_recently_dead += 1; + isdead = false; + break; + default: + elog(ERROR, "unexpected HeapTupleSatisfiesVacuum result"); + isdead = false; /* keep compiler quiet */ + break; + } - if (isdead) - { - *tups_vacuumed += 1; - /* heap rewrite module still needs to see it... */ - if (rewrite_heap_dead_tuple(rwstate, tuple)) + if (isdead) { - /* A previous recently-dead tuple is now known dead */ *tups_vacuumed += 1; - *tups_recently_dead -= 1; + /* heap rewrite module still needs to see it... */ + if (rewrite_heap_dead_tuple(rwstate, tuple)) + { + /* A previous recently-dead tuple is now known dead */ + *tups_vacuumed += 1; + *tups_recently_dead -= 1; + } + + LockBuffer(buf, BUFFER_LOCK_UNLOCK); + continue; } - continue; + + /* + * In the concurrent case, we have a copy of the tuple, so we + * don't worry whether the source tuple will be deleted / updated + * after we release the lock. + */ + LockBuffer(buf, BUFFER_LOCK_UNLOCK); } *num_tuples += 1; @@ -919,7 +950,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, { const int ct_index[] = { PROGRESS_REPACK_HEAP_TUPLES_SCANNED, - PROGRESS_REPACK_HEAP_TUPLES_WRITTEN + PROGRESS_REPACK_HEAP_TUPLES_INSERTED }; int64 ct_val[2]; @@ -934,6 +965,31 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, ct_val[1] = *num_tuples; pgstat_progress_update_multi_param(2, ct_index, ct_val); } + + /* + * Process the WAL produced by the load, as well as by other + * transactions, so that the replication slot can advance and WAL does + * not pile up. Use wal_segment_size as a threshold so that we do not + * introduce the decoding overhead too often. + * + * Of course, we must not apply the changes until the initial load has + * completed. + * + * Note that our insertions into the new table should not be decoded + * as we (intentionally) do not write the logical decoding specific + * information to WAL. + */ + if (concurrent) + { + XLogRecPtr end_of_wal; + + end_of_wal = GetFlushRecPtr(NULL); + if ((end_of_wal - end_of_wal_prev) > wal_segment_size) + { + repack_decode_concurrent_changes(decoding_ctx, end_of_wal); + end_of_wal_prev = end_of_wal; + } + } } if (indexScan != NULL) @@ -977,7 +1033,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, values, isnull, rwstate); /* Report n_tuples */ - pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_WRITTEN, + pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED, n_tuples); } @@ -985,7 +1041,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap, } /* Write out any remaining tuples, and fsync if needed */ - end_heap_rewrite(rwstate); + if (rwstate) + end_heap_rewrite(rwstate); /* Clean up */ pfree(values); @@ -2376,6 +2433,10 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate, * SET WITHOUT OIDS. * * So, we must reconstruct the tuple from component Datums. + * + * If rwstate=NULL, use simple_heap_insert() instead of rewriting - in that + * case we still need to deform/form the tuple. TODO Shouldn't we rename the + * function, as might not do any rewrite? */ static void reform_and_rewrite_tuple(HeapTuple tuple, @@ -2398,8 +2459,28 @@ reform_and_rewrite_tuple(HeapTuple tuple, copiedTuple = heap_form_tuple(newTupDesc, values, isnull); - /* The heap rewrite module does the rest */ - rewrite_heap_tuple(rwstate, tuple, copiedTuple); + if (rwstate) + /* The heap rewrite module does the rest */ + rewrite_heap_tuple(rwstate, tuple, copiedTuple); + else + { + /* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * The following is like simple_heap_insert() except that we pass the + * flag to skip logical decoding: as soon as REPACK CONCURRENTLY swaps + * the relation files, it drops this relation, so no logical + * replication subscription should need the data. + */ + heap_insert(NewHeap, copiedTuple, GetCurrentCommandId(true), + HEAP_INSERT_NO_LOGICAL, NULL); + } heap_freetuple(copiedTuple); } diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c index e6d2b5fced1..6aa2ed214f2 100644 --- a/src/backend/access/heap/rewriteheap.c +++ b/src/backend/access/heap/rewriteheap.c @@ -617,9 +617,9 @@ raw_heap_insert(RewriteState state, HeapTuple tup) int options = HEAP_INSERT_SKIP_FSM; /* - * While rewriting the heap for VACUUM FULL / CLUSTER, make sure data - * for the TOAST table are not logically decoded. The main heap is - * WAL-logged as XLOG FPI records, which are not logically decoded. + * While rewriting the heap for REPACK, make sure data for the TOAST + * table are not logically decoded. The main heap is WAL-logged as + * XLOG FPI records, which are not logically decoded. */ options |= HEAP_INSERT_NO_LOGICAL; diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index b46e7e9c2a6..5670f2bfbde 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -215,6 +215,7 @@ typedef struct TransactionStateData bool parallelChildXact; /* is any parent transaction parallel? */ bool chain; /* start a new block after this one */ bool topXidLogged; /* for a subxact: is top-level XID logged? */ + bool internal; /* for a subxact: launched internally? */ struct TransactionStateData *parent; /* back link to parent */ } TransactionStateData; @@ -4735,6 +4736,7 @@ BeginInternalSubTransaction(const char *name) /* Normal subtransaction start */ PushTransaction(); s = CurrentTransactionState; /* changed by push */ + s->internal = true; /* * Savepoint names, like the TransactionState block itself, live @@ -5251,7 +5253,13 @@ AbortSubTransaction(void) LWLockReleaseAll(); pgstat_report_wait_end(); - pgstat_progress_end_command(); + + /* + * Internal subtransacion might be used by an user command, in which case + * the command outlives the subtransaction. + */ + if (!s->internal) + pgstat_progress_end_command(); pgaio_error_cleanup(); @@ -5468,6 +5476,7 @@ PushTransaction(void) s->parallelModeLevel = 0; s->parallelChildXact = (p->parallelModeLevel != 0 || p->parallelChildXact); s->topXidLogged = false; + s->internal = false; CurrentTransactionState = s; diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index b2b7b10c2be..a92ac78ad9e 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -1266,16 +1266,17 @@ CREATE VIEW pg_stat_progress_cluster AS WHEN 2 THEN 'index scanning heap' WHEN 3 THEN 'sorting tuples' WHEN 4 THEN 'writing new heap' - WHEN 5 THEN 'swapping relation files' - WHEN 6 THEN 'rebuilding index' - WHEN 7 THEN 'performing final cleanup' + -- 5 is 'catch-up', but that should not appear here. + WHEN 6 THEN 'swapping relation files' + WHEN 7 THEN 'rebuilding index' + WHEN 8 THEN 'performing final cleanup' END AS phase, CAST(S.param3 AS oid) AS cluster_index_relid, S.param4 AS heap_tuples_scanned, S.param5 AS heap_tuples_written, - S.param6 AS heap_blks_total, - S.param7 AS heap_blks_scanned, - S.param8 AS index_rebuild_count + S.param8 AS heap_blks_total, + S.param9 AS heap_blks_scanned, + S.param10 AS index_rebuild_count FROM pg_stat_get_progress_info('CLUSTER') AS S LEFT JOIN pg_database D ON S.datid = D.oid; @@ -1291,16 +1292,19 @@ CREATE VIEW pg_stat_progress_repack AS WHEN 2 THEN 'index scanning heap' WHEN 3 THEN 'sorting tuples' WHEN 4 THEN 'writing new heap' - WHEN 5 THEN 'swapping relation files' - WHEN 6 THEN 'rebuilding index' - WHEN 7 THEN 'performing final cleanup' + WHEN 5 THEN 'catch-up' + WHEN 6 THEN 'swapping relation files' + WHEN 7 THEN 'rebuilding index' + WHEN 8 THEN 'performing final cleanup' END AS phase, CAST(S.param3 AS oid) AS repack_index_relid, S.param4 AS heap_tuples_scanned, - S.param5 AS heap_tuples_written, - S.param6 AS heap_blks_total, - S.param7 AS heap_blks_scanned, - S.param8 AS index_rebuild_count + S.param5 AS heap_tuples_inserted, + S.param6 AS heap_tuples_updated, + S.param7 AS heap_tuples_deleted, + S.param8 AS heap_blks_total, + S.param9 AS heap_blks_scanned, + S.param10 AS index_rebuild_count FROM pg_stat_get_progress_info('REPACK') AS S LEFT JOIN pg_database D ON S.datid = D.oid; diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c index af14a230f94..aa3ae85bcee 100644 --- a/src/backend/commands/cluster.c +++ b/src/backend/commands/cluster.c @@ -25,6 +25,10 @@ #include "access/toast_internals.h" #include "access/transam.h" #include "access/xact.h" +#include "access/xlog.h" +#include "access/xlog_internal.h" +#include "access/xloginsert.h" +#include "access/xlogutils.h" #include "catalog/catalog.h" #include "catalog/dependency.h" #include "catalog/heap.h" @@ -32,6 +36,7 @@ #include "catalog/namespace.h" #include "catalog/objectaccess.h" #include "catalog/pg_am.h" +#include "catalog/pg_control.h" #include "catalog/pg_inherits.h" #include "catalog/toasting.h" #include "commands/cluster.h" @@ -39,15 +44,21 @@ #include "commands/progress.h" #include "commands/tablecmds.h" #include "commands/vacuum.h" +#include "executor/executor.h" #include "miscadmin.h" #include "optimizer/optimizer.h" #include "pgstat.h" +#include "replication/decode.h" +#include "replication/logical.h" +#include "replication/snapbuild.h" #include "storage/bufmgr.h" +#include "storage/ipc.h" #include "storage/lmgr.h" #include "storage/predicate.h" #include "utils/acl.h" #include "utils/fmgroids.h" #include "utils/guc.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -67,13 +78,45 @@ typedef struct Oid indexOid; } RelToCluster; +/* + * The following definitions are used for concurrent processing. + */ + +/* + * The locators are used to avoid logical decoding of data that we do not need + * for our table. + */ +RelFileLocator repacked_rel_locator = {.relNumber = InvalidOid}; +RelFileLocator repacked_rel_toast_locator = {.relNumber = InvalidOid}; + +/* + * Everything we need to call ExecInsertIndexTuples(). + */ +typedef struct IndexInsertState +{ + ResultRelInfo *rri; + EState *estate; + + Relation ident_index; +} IndexInsertState; + +/* The WAL segment being decoded. */ +static XLogSegNo repack_current_segment = 0; + + static bool cluster_rel_recheck(RepackCommand cmd, Relation OldHeap, - Oid indexOid, Oid userid, int options); + Oid indexOid, Oid userid, LOCKMODE lmode, + int options); +static void check_repack_concurrently_requirements(Relation rel); static void rebuild_relation(RepackCommand cmd, bool usingindex, - Relation OldHeap, Relation index, bool verbose); + Relation OldHeap, Relation index, Oid userid, + bool verbose, bool concurrent); static void copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex, - bool verbose, bool *pSwapToastByContent, - TransactionId *pFreezeXid, MultiXactId *pCutoffMulti); + Snapshot snapshot, LogicalDecodingContext *decoding_ctx, + bool verbose, + bool *pSwapToastByContent, + TransactionId *pFreezeXid, + MultiXactId *pCutoffMulti); static List *get_tables_to_repack(RepackCommand cmd, bool usingindex, MemoryContext permcxt); static List *get_tables_to_repack_partitioned(RepackCommand cmd, @@ -81,12 +124,61 @@ static List *get_tables_to_repack_partitioned(RepackCommand cmd, Oid relid, bool rel_is_index); static bool cluster_is_permitted_for_relation(RepackCommand cmd, Oid relid, Oid userid); + +static void begin_concurrent_repack(Relation rel); +static void end_concurrent_repack(void); +static LogicalDecodingContext *setup_logical_decoding(Oid relid, + const char *slotname, + TupleDesc tupdesc); +static HeapTuple get_changed_tuple(char *change); +static void apply_concurrent_changes(RepackDecodingState *dstate, + Relation rel, ScanKey key, int nkeys, + IndexInsertState *iistate); +static void apply_concurrent_insert(Relation rel, ConcurrentChange *change, + HeapTuple tup, IndexInsertState *iistate, + TupleTableSlot *index_slot); +static void apply_concurrent_update(Relation rel, HeapTuple tup, + HeapTuple tup_target, + ConcurrentChange *change, + IndexInsertState *iistate, + TupleTableSlot *index_slot); +static void apply_concurrent_delete(Relation rel, HeapTuple tup_target, + ConcurrentChange *change); +static HeapTuple find_target_tuple(Relation rel, ScanKey key, int nkeys, + HeapTuple tup_key, + IndexInsertState *iistate, + TupleTableSlot *ident_slot, + IndexScanDesc *scan_p); +static void process_concurrent_changes(LogicalDecodingContext *ctx, + XLogRecPtr end_of_wal, + Relation rel_dst, + Relation rel_src, + ScanKey ident_key, + int ident_key_nentries, + IndexInsertState *iistate); +static IndexInsertState *get_index_insert_state(Relation relation, + Oid ident_index_id); +static ScanKey build_identity_key(Oid ident_idx_oid, Relation rel_src, + int *nentries); +static void free_index_insert_state(IndexInsertState *iistate); +static void cleanup_logical_decoding(LogicalDecodingContext *ctx); +static void rebuild_relation_finish_concurrent(Relation NewHeap, Relation OldHeap, + Relation cl_index, + LogicalDecodingContext *ctx, + bool swap_toast_by_content, + TransactionId frozenXid, + MultiXactId cutoffMulti); +static List *build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes); static Relation process_single_relation(RepackStmt *stmt, + LOCKMODE lockmode, + bool isTopLevel, ClusterParams *params); static Oid determine_clustered_index(Relation rel, bool usingindex, const char *indexname); +#define REPL_PLUGIN_NAME "pgoutput_repack" + static const char * RepackCommandAsString(RepackCommand cmd) { @@ -95,9 +187,9 @@ RepackCommandAsString(RepackCommand cmd) case REPACK_COMMAND_REPACK: return "REPACK"; case REPACK_COMMAND_VACUUMFULL: - return "VACUUM"; + return "VACUUM (FULL)"; case REPACK_COMMAND_CLUSTER: - return "VACUUM"; + return "CLUSTER"; } return "???"; } @@ -130,16 +222,27 @@ void ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel) { ClusterParams params = {0}; - bool verbose = false; Relation rel = NULL; MemoryContext repack_context; + LOCKMODE lockmode; List *rtcs; /* Parse option list */ foreach_node(DefElem, opt, stmt->params) { - if (strcmp(opt->defname, "verbose") == 0) - verbose = defGetBoolean(opt); + if (strcmp(opt->defname, "verbose") == 0 && + defGetBoolean(opt)) + params.options |= CLUOPT_VERBOSE; + else if (strcmp(opt->defname, "concurrently") == 0 && + defGetBoolean(opt)) + { + if (stmt->command != REPACK_COMMAND_REPACK) + ereport(ERROR, + errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("CONCURRENTLY option not supported for %s", + RepackCommandAsString(stmt->command))); + params.options |= CLUOPT_CONCURRENT; + } else ereport(ERROR, (errcode(ERRCODE_SYNTAX_ERROR), @@ -149,7 +252,17 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel) parser_errposition(pstate, opt->location))); } - params.options = (verbose ? CLUOPT_VERBOSE : 0); + /* + * Determine the lock mode expected by cluster_rel(). + * + * In the exclusive case, we obtain AccessExclusiveLock right away to + * avoid lock-upgrade hazard in the single-transaction case. In the + * CONCURRENTLY case, the AccessExclusiveLock will only be used at the end + * of processing, supposedly for very short time. Until then, we'll have + * to unlock the relation temporarily, so there's no lock-upgrade hazard. + */ + lockmode = (params.options & CLUOPT_CONCURRENT) == 0 ? + AccessExclusiveLock : ShareUpdateExclusiveLock; /* * If a single relation is specified, process it and we're done ... unless @@ -157,16 +270,35 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel) */ if (stmt->relation != NULL) { - rel = process_single_relation(stmt, ¶ms); + rel = process_single_relation(stmt, lockmode, isTopLevel, ¶ms); if (rel == NULL) return; } /* - * By here, we know we are in a multi-table situation. In order to avoid - * holding locks for too long, we want to process each table in its own - * transaction. This forces us to disallow running inside a user - * transaction block. + * By here, we know we are in a multi-table situation. + * + * Concurrent processing is currently considered rather special (e.g. in + * terms of resources consumed) so it is not performed in bulk. + */ + if (params.options & CLUOPT_CONCURRENT) + { + if (rel != NULL) + { + Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE); + ereport(ERROR, + errmsg("REPACK CONCURRENTLY not supported for partitioned tables"), + errhint("Consider running the command for individual partitions.")); + } + else + ereport(ERROR, + errmsg("REPACK CONCURRENTLY requires explicit table name")); + } + + /* + * In order to avoid holding locks for too long, we want to process each + * table in its own transaction. This forces us to disallow running + * inside a user transaction block. */ PreventInTransactionBlock(isTopLevel, RepackCommandAsString(stmt->command)); @@ -247,13 +379,13 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel) * Open the target table, coping with the case where it has been * dropped. */ - rel = try_table_open(rtc->tableOid, AccessExclusiveLock); + rel = try_table_open(rtc->tableOid, lockmode); if (rel == NULL) continue; /* Process this table */ cluster_rel(stmt->command, stmt->usingindex, - rel, rtc->indexOid, ¶ms); + rel, rtc->indexOid, ¶ms, isTopLevel); /* cluster_rel closes the relation, but keeps lock */ PopActiveSnapshot(); @@ -282,22 +414,55 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel) * If indexOid is InvalidOid, the table will be rewritten in physical order * instead of index order. * + * Note that, in the concurrent case, the function releases the lock at some + * point, in order to get AccessExclusiveLock for the final steps (i.e. to + * swap the relation files). To make things simpler, the caller should expect + * OldHeap to be closed on return, regardless CLUOPT_CONCURRENT. (The + * AccessExclusiveLock is kept till the end of the transaction.) + * * 'cmd' indicates which command is being executed, to be used for error * messages. */ void cluster_rel(RepackCommand cmd, bool usingindex, - Relation OldHeap, Oid indexOid, ClusterParams *params) + Relation OldHeap, Oid indexOid, ClusterParams *params, + bool isTopLevel) { Oid tableOid = RelationGetRelid(OldHeap); + Relation index; + LOCKMODE lmode; Oid save_userid; int save_sec_context; int save_nestlevel; bool verbose = ((params->options & CLUOPT_VERBOSE) != 0); bool recheck = ((params->options & CLUOPT_RECHECK) != 0); - Relation index; + bool concurrent = ((params->options & CLUOPT_CONCURRENT) != 0); + + /* + * Check that the correct lock is held. The lock mode is + * AccessExclusiveLock for normal processing and ShareUpdateExclusiveLock + * for concurrent processing (so that SELECT, INSERT, UPDATE and DELETE + * commands work, but cluster_rel() cannot be called concurrently for the + * same relation). + */ + lmode = !concurrent ? AccessExclusiveLock : ShareUpdateExclusiveLock; + + /* There are specific requirements on concurrent processing. */ + if (concurrent) + { + /* + * Make sure we have no XID assigned, otherwise call of + * setup_logical_decoding() can cause a deadlock. + * + * The existence of transaction block actually does not imply that XID + * was already assigned, but it very likely is. We might want to check + * the result of GetCurrentTransactionIdIfAny() instead, but that + * would be less clear from user's perspective. + */ + PreventInTransactionBlock(isTopLevel, "REPACK (CONCURRENTLY)"); - Assert(CheckRelationLockedByMe(OldHeap, AccessExclusiveLock, false)); + check_repack_concurrently_requirements(OldHeap); + } /* Check for user-requested abort. */ CHECK_FOR_INTERRUPTS(); @@ -340,11 +505,13 @@ cluster_rel(RepackCommand cmd, bool usingindex, * If this is a single-transaction CLUSTER, we can skip these tests. We * *must* skip the one on indisclustered since it would reject an attempt * to cluster a not-previously-clustered index. + * + * XXX move [some of] these comments to where the RECHECK flag is + * determined? */ - if (recheck) - if (!cluster_rel_recheck(cmd, OldHeap, indexOid, save_userid, - params->options)) - goto out; + if (recheck && !cluster_rel_recheck(cmd, OldHeap, indexOid, save_userid, + lmode, params->options)) + goto out; /* * We allow repacking shared catalogs only when not using an index. It @@ -358,6 +525,12 @@ cluster_rel(RepackCommand cmd, bool usingindex, errmsg("cannot run \"%s\" on a shared catalog", RepackCommandAsString(cmd)))); + /* + * The CONCURRENTLY case should have been rejected earlier because it does + * not support system catalogs. + */ + Assert(!(OldHeap->rd_rel->relisshared && concurrent)); + /* * Don't process temp tables of other backends ... their local buffer * manager is not going to cope. @@ -393,7 +566,7 @@ cluster_rel(RepackCommand cmd, bool usingindex, if (OidIsValid(indexOid)) { /* verify the index is good and lock it */ - check_index_is_clusterable(OldHeap, indexOid, AccessExclusiveLock); + check_index_is_clusterable(OldHeap, indexOid, lmode); /* also open it */ index = index_open(indexOid, NoLock); } @@ -410,7 +583,9 @@ cluster_rel(RepackCommand cmd, bool usingindex, if (OldHeap->rd_rel->relkind == RELKIND_MATVIEW && !RelationIsPopulated(OldHeap)) { - relation_close(OldHeap, AccessExclusiveLock); + if (index) + index_close(index, lmode); + relation_close(OldHeap, lmode); goto out; } @@ -423,11 +598,35 @@ cluster_rel(RepackCommand cmd, bool usingindex, * invalid, because we move tuples around. Promote them to relation * locks. Predicate locks on indexes will be promoted when they are * reindexed. + * + * During concurrent processing, the heap as well as its indexes stay in + * operation, so we postpone this step until they are locked using + * AccessExclusiveLock near the end of the processing. */ - TransferPredicateLocksToHeapRelation(OldHeap); + if (!concurrent) + TransferPredicateLocksToHeapRelation(OldHeap); /* rebuild_relation does all the dirty work */ - rebuild_relation(cmd, usingindex, OldHeap, index, verbose); + PG_TRY(); + { + /* + * For concurrent processing, make sure that our logical decoding + * ignores data changes of other tables than the one we are + * processing. + */ + if (concurrent) + begin_concurrent_repack(OldHeap); + + rebuild_relation(cmd, usingindex, OldHeap, index, save_userid, + verbose, concurrent); + } + PG_FINALLY(); + { + if (concurrent) + end_concurrent_repack(); + } + PG_END_TRY(); + /* rebuild_relation closes OldHeap, and index if valid */ out: @@ -446,14 +645,14 @@ out: */ static bool cluster_rel_recheck(RepackCommand cmd, Relation OldHeap, Oid indexOid, - Oid userid, int options) + Oid userid, LOCKMODE lmode, int options) { Oid tableOid = RelationGetRelid(OldHeap); /* Check that the user still has privileges for the relation */ if (!cluster_is_permitted_for_relation(cmd, tableOid, userid)) { - relation_close(OldHeap, AccessExclusiveLock); + relation_close(OldHeap, lmode); return false; } @@ -467,7 +666,7 @@ cluster_rel_recheck(RepackCommand cmd, Relation OldHeap, Oid indexOid, */ if (RELATION_IS_OTHER_TEMP(OldHeap)) { - relation_close(OldHeap, AccessExclusiveLock); + relation_close(OldHeap, lmode); return false; } @@ -478,7 +677,7 @@ cluster_rel_recheck(RepackCommand cmd, Relation OldHeap, Oid indexOid, */ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(indexOid))) { - relation_close(OldHeap, AccessExclusiveLock); + relation_close(OldHeap, lmode); return false; } @@ -489,7 +688,7 @@ cluster_rel_recheck(RepackCommand cmd, Relation OldHeap, Oid indexOid, if ((options & CLUOPT_RECHECK_ISCLUSTERED) != 0 && !get_index_isclustered(indexOid)) { - relation_close(OldHeap, AccessExclusiveLock); + relation_close(OldHeap, lmode); return false; } } @@ -630,19 +829,89 @@ mark_index_clustered(Relation rel, Oid indexOid, bool is_internal) table_close(pg_index, RowExclusiveLock); } +/* + * Check if the CONCURRENTLY option is legal for the relation. + */ +static void +check_repack_concurrently_requirements(Relation rel) +{ + char relpersistence, + replident; + Oid ident_idx; + + /* Data changes in system relations are not logically decoded. */ + if (IsCatalogRelation(rel)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot repack relation \"%s\"", + RelationGetRelationName(rel)), + errhint("REPACK CONCURRENTLY is not supported for catalog relations."))); + + /* + * reorderbuffer.c does not seem to handle processing of TOAST relation + * alone. + */ + if (IsToastRelation(rel)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot repack relation \"%s\"", + RelationGetRelationName(rel)), + errhint("REPACK CONCURRENTLY is not supported for TOAST relations, unless the main relation is repacked too."))); + + relpersistence = rel->rd_rel->relpersistence; + if (relpersistence != RELPERSISTENCE_PERMANENT) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot repack relation \"%s\"", + RelationGetRelationName(rel)), + errhint("REPACK CONCURRENTLY is only allowed for permanent relations."))); + + /* With NOTHING, WAL does not contain the old tuple. */ + replident = rel->rd_rel->relreplident; + if (replident == REPLICA_IDENTITY_NOTHING) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot repack relation \"%s\"", + RelationGetRelationName(rel)), + errhint("Relation \"%s\" has insufficient replication identity.", + RelationGetRelationName(rel)))); + + /* + * Identity index is not set if the replica identity is FULL, but PK might + * exist in such a case. + */ + ident_idx = RelationGetReplicaIndex(rel); + if (!OidIsValid(ident_idx) && OidIsValid(rel->rd_pkindex)) + ident_idx = rel->rd_pkindex; + if (!OidIsValid(ident_idx)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot process relation \"%s\"", + RelationGetRelationName(rel)), + (errhint("Relation \"%s\" has no identity index.", + RelationGetRelationName(rel))))); +} + + /* * rebuild_relation: rebuild an existing relation in index or physical order * - * OldHeap: table to rebuild. + * OldHeap: table to rebuild. See cluster_rel() for comments on the required + * lock strength. + * * index: index to cluster by, or NULL to rewrite in physical order. * - * On entry, heap and index (if one is given) must be open, and - * AccessExclusiveLock held on them. - * On exit, they are closed, but locks on them are not released. + * On entry, heap and index (if one is given) must be open, and the + * appropriate lock held on them -- AccessExclusiveLock for exclusive + * processing and ShareUpdateExclusiveLock for concurrent processing. + * + * On exit, they are closed, but still locked with AccessExclusiveLock. (The + * function handles the lock upgrade if 'concurrent' is true.) */ static void rebuild_relation(RepackCommand cmd, bool usingindex, - Relation OldHeap, Relation index, bool verbose) + Relation OldHeap, Relation index, Oid userid, + bool verbose, bool concurrent) { Oid tableOid = RelationGetRelid(OldHeap); Oid accessMethod = OldHeap->rd_rel->relam; @@ -650,13 +919,55 @@ rebuild_relation(RepackCommand cmd, bool usingindex, Oid OIDNewHeap; Relation NewHeap; char relpersistence; - bool is_system_catalog; bool swap_toast_by_content; TransactionId frozenXid; MultiXactId cutoffMulti; + NameData slotname; + LogicalDecodingContext *ctx = NULL; + Snapshot snapshot = NULL; +#if USE_ASSERT_CHECKING + LOCKMODE lmode; + + lmode = concurrent ? ShareUpdateExclusiveLock : AccessExclusiveLock; + + Assert(CheckRelationLockedByMe(OldHeap, lmode, false)); + Assert(!usingindex || CheckRelationLockedByMe(index, lmode, false)); +#endif + + if (concurrent) + { + TupleDesc tupdesc; + + /* + * REPACK CONCURRENTLY is not allowed in a transaction block, so this + * should never fire. + */ + Assert(GetTopTransactionIdIfAny() == InvalidTransactionId); + + /* + * A single backend should not execute multiple REPACK commands at a + * time, so use PID to make the slot unique. + */ + snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); + + tupdesc = CreateTupleDescCopy(RelationGetDescr(OldHeap)); + + /* + * Prepare to capture the concurrent data changes. + * + * Note that this call waits for all transactions with XID already + * assigned to finish. If some of those transactions is waiting for a + * lock conflicting with ShareUpdateExclusiveLock on our table (e.g. + * it runs CREATE INDEX), we can end up in a deadlock. Not sure this + * risk is worth unlocking/locking the table (and its clustering + * index) and checking again if its still eligible for REPACK + * CONCURRENTLY. + */ + ctx = setup_logical_decoding(tableOid, NameStr(slotname), tupdesc); - Assert(CheckRelationLockedByMe(OldHeap, AccessExclusiveLock, false) && - (index == NULL || CheckRelationLockedByMe(index, AccessExclusiveLock, false))); + snapshot = SnapBuildInitialSnapshotForRepack(ctx->snapshot_builder); + PushActiveSnapshot(snapshot); + } /* for CLUSTER or REPACK USING INDEX, mark the index as the one to use */ if (usingindex) @@ -664,7 +975,6 @@ rebuild_relation(RepackCommand cmd, bool usingindex, /* Remember info about rel before closing OldHeap */ relpersistence = OldHeap->rd_rel->relpersistence; - is_system_catalog = IsSystemRelation(OldHeap); /* * Create the transient table that will receive the re-ordered data. @@ -680,30 +990,67 @@ rebuild_relation(RepackCommand cmd, bool usingindex, NewHeap = table_open(OIDNewHeap, NoLock); /* Copy the heap data into the new table in the desired order */ - copy_table_data(NewHeap, OldHeap, index, verbose, + copy_table_data(NewHeap, OldHeap, index, snapshot, ctx, verbose, &swap_toast_by_content, &frozenXid, &cutoffMulti); + /* The historic snapshot won't be needed anymore. */ + if (snapshot) + PopActiveSnapshot(); - /* Close relcache entries, but keep lock until transaction commit */ - table_close(OldHeap, NoLock); - if (index) - index_close(index, NoLock); + if (concurrent) + { + /* + * Push a snapshot that we will use to find old versions of rows when + * processing concurrent UPDATE and DELETE commands. (That snapshot + * should also be used by index expressions.) + */ + PushActiveSnapshot(GetTransactionSnapshot()); - /* - * Close the new relation so it can be dropped as soon as the storage is - * swapped. The relation is not visible to others, so no need to unlock it - * explicitly. - */ - table_close(NewHeap, NoLock); + /* + * Make sure we can find the tuples just inserted when applying DML + * commands on top of those. + */ + CommandCounterIncrement(); + UpdateActiveSnapshotCommandId(); - /* - * Swap the physical files of the target and transient tables, then - * rebuild the target's indexes and throw away the transient table. - */ - finish_heap_swap(tableOid, OIDNewHeap, is_system_catalog, - swap_toast_by_content, false, true, - frozenXid, cutoffMulti, - relpersistence); + rebuild_relation_finish_concurrent(NewHeap, OldHeap, index, + ctx, swap_toast_by_content, + frozenXid, cutoffMulti); + PopActiveSnapshot(); + + pgstat_progress_update_param(PROGRESS_REPACK_PHASE, + PROGRESS_REPACK_PHASE_FINAL_CLEANUP); + + /* Done with decoding. */ + cleanup_logical_decoding(ctx); + ReplicationSlotRelease(); + ReplicationSlotDrop(NameStr(slotname), false); + } + else + { + bool is_system_catalog = IsSystemRelation(OldHeap); + + /* Close relcache entries, but keep lock until transaction commit */ + table_close(OldHeap, NoLock); + if (index) + index_close(index, NoLock); + + /* + * Close the new relation so it can be dropped as soon as the storage + * is swapped. The relation is not visible to others, so no need to + * unlock it explicitly. + */ + table_close(NewHeap, NoLock); + + /* + * Swap the physical files of the target and transient tables, then + * rebuild the target's indexes and throw away the transient table. + */ + finish_heap_swap(tableOid, OIDNewHeap, is_system_catalog, + swap_toast_by_content, false, true, true, + frozenXid, cutoffMulti, + relpersistence); + } } @@ -838,15 +1185,19 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, /* * Do the physical copying of table data. * + * 'snapshot' and 'decoding_ctx': see table_relation_copy_for_cluster(). Pass + * iff concurrent processing is required. + * * There are three output parameters: * *pSwapToastByContent is set true if toast tables must be swapped by content. * *pFreezeXid receives the TransactionId used as freeze cutoff point. * *pCutoffMulti receives the MultiXactId used as a cutoff point. */ static void -copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex, bool verbose, - bool *pSwapToastByContent, TransactionId *pFreezeXid, - MultiXactId *pCutoffMulti) +copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex, + Snapshot snapshot, LogicalDecodingContext *decoding_ctx, + bool verbose, bool *pSwapToastByContent, + TransactionId *pFreezeXid, MultiXactId *pCutoffMulti) { Relation relRelation; HeapTuple reltup; @@ -864,6 +1215,8 @@ copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex, bool verb PGRUsage ru0; char *nspname; + bool concurrent = snapshot != NULL; + pg_rusage_init(&ru0); /* Store a copy of the namespace name for logging purposes */ @@ -966,8 +1319,48 @@ copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex, bool verb * provided, else plain seqscan. */ if (OldIndex != NULL && OldIndex->rd_rel->relam == BTREE_AM_OID) + { + ResourceOwner oldowner = NULL; + ResourceOwner resowner = NULL; + + /* + * In the CONCURRENT case, use a dedicated resource owner so we don't + * leave any additional locks behind us that we cannot release easily. + */ + if (concurrent) + { + Assert(CheckRelationLockedByMe(OldHeap, ShareUpdateExclusiveLock, + false)); + Assert(CheckRelationLockedByMe(OldIndex, ShareUpdateExclusiveLock, + false)); + + resowner = ResourceOwnerCreate(CurrentResourceOwner, + "plan_cluster_use_sort"); + oldowner = CurrentResourceOwner; + CurrentResourceOwner = resowner; + } + use_sort = plan_cluster_use_sort(RelationGetRelid(OldHeap), RelationGetRelid(OldIndex)); + + if (concurrent) + { + CurrentResourceOwner = oldowner; + + /* + * We are primarily concerned about locks, but if the planner + * happened to allocate any other resources, we should release + * them too because we're going to delete the whole resowner. + */ + ResourceOwnerRelease(resowner, RESOURCE_RELEASE_BEFORE_LOCKS, + false, false); + ResourceOwnerRelease(resowner, RESOURCE_RELEASE_LOCKS, + false, false); + ResourceOwnerRelease(resowner, RESOURCE_RELEASE_AFTER_LOCKS, + false, false); + ResourceOwnerDelete(resowner); + } + } else use_sort = false; @@ -996,7 +1389,9 @@ copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex, bool verb * values (e.g. because the AM doesn't use freezing). */ table_relation_copy_for_cluster(OldHeap, NewHeap, OldIndex, use_sort, - cutoffs.OldestXmin, &cutoffs.FreezeLimit, + cutoffs.OldestXmin, snapshot, + decoding_ctx, + &cutoffs.FreezeLimit, &cutoffs.MultiXactCutoff, &num_tuples, &tups_vacuumed, &tups_recently_dead); @@ -1005,7 +1400,11 @@ copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex, bool verb *pFreezeXid = cutoffs.FreezeLimit; *pCutoffMulti = cutoffs.MultiXactCutoff; - /* Reset rd_toastoid just to be tidy --- it shouldn't be looked at again */ + /* + * Reset rd_toastoid just to be tidy --- it shouldn't be looked at again. + * In the CONCURRENTLY case, we need to set it again before applying the + * concurrent changes. + */ NewHeap->rd_toastoid = InvalidOid; num_pages = RelationGetNumberOfBlocks(NewHeap); @@ -1463,14 +1862,13 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool swap_toast_by_content, bool check_constraints, bool is_internal, + bool reindex, TransactionId frozenXid, MultiXactId cutoffMulti, char newrelpersistence) { ObjectAddress object; Oid mapped_tables[4]; - int reindex_flags; - ReindexParams reindex_params = {0}; int i; /* Report that we are now swapping relation files */ @@ -1496,39 +1894,47 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, if (is_system_catalog) CacheInvalidateCatalog(OIDOldHeap); - /* - * Rebuild each index on the relation (but not the toast table, which is - * all-new at this point). It is important to do this before the DROP - * step because if we are processing a system catalog that will be used - * during DROP, we want to have its indexes available. There is no - * advantage to the other order anyway because this is all transactional, - * so no chance to reclaim disk space before commit. We do not need a - * final CommandCounterIncrement() because reindex_relation does it. - * - * Note: because index_build is called via reindex_relation, it will never - * set indcheckxmin true for the indexes. This is OK even though in some - * sense we are building new indexes rather than rebuilding existing ones, - * because the new heap won't contain any HOT chains at all, let alone - * broken ones, so it can't be necessary to set indcheckxmin. - */ - reindex_flags = REINDEX_REL_SUPPRESS_INDEX_USE; - if (check_constraints) - reindex_flags |= REINDEX_REL_CHECK_CONSTRAINTS; + if (reindex) + { + int reindex_flags; + ReindexParams reindex_params = {0}; - /* - * Ensure that the indexes have the same persistence as the parent - * relation. - */ - if (newrelpersistence == RELPERSISTENCE_UNLOGGED) - reindex_flags |= REINDEX_REL_FORCE_INDEXES_UNLOGGED; - else if (newrelpersistence == RELPERSISTENCE_PERMANENT) - reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT; + /* + * Rebuild each index on the relation (but not the toast table, which + * is all-new at this point). It is important to do this before the + * DROP step because if we are processing a system catalog that will + * be used during DROP, we want to have its indexes available. There + * is no advantage to the other order anyway because this is all + * transactional, so no chance to reclaim disk space before commit. We + * do not need a final CommandCounterIncrement() because + * reindex_relation does it. + * + * Note: because index_build is called via reindex_relation, it will + * never set indcheckxmin true for the indexes. This is OK even + * though in some sense we are building new indexes rather than + * rebuilding existing ones, because the new heap won't contain any + * HOT chains at all, let alone broken ones, so it can't be necessary + * to set indcheckxmin. + */ + reindex_flags = REINDEX_REL_SUPPRESS_INDEX_USE; + if (check_constraints) + reindex_flags |= REINDEX_REL_CHECK_CONSTRAINTS; - /* Report that we are now reindexing relations */ - pgstat_progress_update_param(PROGRESS_REPACK_PHASE, - PROGRESS_REPACK_PHASE_REBUILD_INDEX); + /* + * Ensure that the indexes have the same persistence as the parent + * relation. + */ + if (newrelpersistence == RELPERSISTENCE_UNLOGGED) + reindex_flags |= REINDEX_REL_FORCE_INDEXES_UNLOGGED; + else if (newrelpersistence == RELPERSISTENCE_PERMANENT) + reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT; - reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params); + /* Report that we are now reindexing relations */ + pgstat_progress_update_param(PROGRESS_REPACK_PHASE, + PROGRESS_REPACK_PHASE_REBUILD_INDEX); + + reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params); + } /* Report that we are now doing clean up */ pgstat_progress_update_param(PROGRESS_REPACK_PHASE, @@ -1870,7 +2276,8 @@ cluster_is_permitted_for_relation(RepackCommand cmd, Oid relid, Oid userid) * resolve in this case. */ static Relation -process_single_relation(RepackStmt *stmt, ClusterParams *params) +process_single_relation(RepackStmt *stmt, LOCKMODE lockmode, bool isTopLevel, + ClusterParams *params) { Relation rel; Oid tableOid; @@ -1879,13 +2286,9 @@ process_single_relation(RepackStmt *stmt, ClusterParams *params) Assert(stmt->command == REPACK_COMMAND_CLUSTER || stmt->command == REPACK_COMMAND_REPACK); - /* - * Find, lock, and check permissions on the table. We obtain - * AccessExclusiveLock right away to avoid lock-upgrade hazard in the - * single-transaction case. - */ + /* Find, lock, and check permissions on the table. */ tableOid = RangeVarGetRelidExtended(stmt->relation, - AccessExclusiveLock, + lockmode, 0, RangeVarCallbackMaintainsTable, NULL); @@ -1911,12 +2314,17 @@ process_single_relation(RepackStmt *stmt, ClusterParams *params) return rel; else { - Oid indexOid; + Oid indexOid = InvalidOid; + + if (stmt->usingindex) + { + indexOid = determine_clustered_index(rel, stmt->usingindex, + stmt->indexname); + check_index_is_clusterable(rel, indexOid, lockmode); + } - indexOid = determine_clustered_index(rel, stmt->usingindex, - stmt->indexname); - check_index_is_clusterable(rel, indexOid, AccessExclusiveLock); - cluster_rel(stmt->command, stmt->usingindex, rel, indexOid, params); + cluster_rel(stmt->command, stmt->usingindex, rel, indexOid, + params, isTopLevel); return NULL; } } @@ -1973,3 +2381,1052 @@ determine_clustered_index(Relation rel, bool usingindex, const char *indexname) return indexOid; } + + +/* + * Call this function before REPACK CONCURRENTLY starts to setup logical + * decoding. It makes sure that other users of the table put enough + * information into WAL. + * + * The point is that at various places we expect that the table we're + * processing is treated like a system catalog. For example, we need to be + * able to scan it using a "historic snapshot" anytime during the processing + * (as opposed to scanning only at the start point of the decoding, as logical + * replication does during initial table synchronization), in order to apply + * concurrent UPDATE / DELETE commands. + * + * Note that TOAST table needs no attention here as it's not scanned using + * historic snapshot. + */ +static void +begin_concurrent_repack(Relation rel) +{ + Oid toastrelid; + + /* Avoid logical decoding of other relations by this backend. */ + repacked_rel_locator = rel->rd_locator; + toastrelid = rel->rd_rel->reltoastrelid; + if (OidIsValid(toastrelid)) + { + Relation toastrel; + + /* Avoid logical decoding of other TOAST relations. */ + toastrel = table_open(toastrelid, AccessShareLock); + repacked_rel_toast_locator = toastrel->rd_locator; + table_close(toastrel, AccessShareLock); + } +} + +/* + * Call this when done with REPACK CONCURRENTLY. + */ +static void +end_concurrent_repack(void) +{ + /* + * Restore normal function of (future) logical decoding for this backend. + */ + repacked_rel_locator.relNumber = InvalidOid; + repacked_rel_toast_locator.relNumber = InvalidOid; +} + +/* + * This function is much like pg_create_logical_replication_slot() except that + * the new slot is neither released (if anyone else could read changes from + * our slot, we could miss changes other backends do while we copy the + * existing data into temporary table), nor persisted (it's easier to handle + * crash by restarting all the work from scratch). + */ +static LogicalDecodingContext * +setup_logical_decoding(Oid relid, const char *slotname, TupleDesc tupdesc) +{ + LogicalDecodingContext *ctx; + RepackDecodingState *dstate; + + /* + * Check if we can use logical decoding. + */ + CheckSlotPermissions(); + CheckLogicalDecodingRequirements(); + + /* RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ + ReplicationSlotCreate(slotname, true, RS_TEMPORARY, false, false, false); + + /* + * Neither prepare_write nor do_write callback nor update_progress is + * useful for us. + * + * Regarding the value of need_full_snapshot, we pass false because the + * table we are processing is present in RepackedRelsHash and therefore, + * regarding logical decoding, treated like a catalog. + */ + ctx = CreateInitDecodingContext(REPL_PLUGIN_NAME, + NIL, + false, + InvalidXLogRecPtr, + XL_ROUTINE(.page_read = read_local_xlog_page, + .segment_open = wal_segment_open, + .segment_close = wal_segment_close), + NULL, NULL, NULL); + + /* + * We don't have control on setting fast_forward, so at least check it. + */ + Assert(!ctx->fast_forward); + + DecodingContextFindStartpoint(ctx); + + /* Some WAL records should have been read. */ + Assert(ctx->reader->EndRecPtr != InvalidXLogRecPtr); + + XLByteToSeg(ctx->reader->EndRecPtr, repack_current_segment, + wal_segment_size); + + /* + * Setup structures to store decoded changes. + */ + dstate = palloc0(sizeof(RepackDecodingState)); + dstate->relid = relid; + dstate->tstore = tuplestore_begin_heap(false, false, + maintenance_work_mem); + + dstate->tupdesc = tupdesc; + + /* Initialize the descriptor to store the changes ... */ + dstate->tupdesc_change = CreateTemplateTupleDesc(1); + + TupleDescInitEntry(dstate->tupdesc_change, 1, NULL, BYTEAOID, -1, 0); + /* ... as well as the corresponding slot. */ + dstate->tsslot = MakeSingleTupleTableSlot(dstate->tupdesc_change, + &TTSOpsMinimalTuple); + + dstate->resowner = ResourceOwnerCreate(CurrentResourceOwner, + "logical decoding"); + + ctx->output_writer_private = dstate; + return ctx; +} + +/* + * Retrieve tuple from ConcurrentChange structure. + * + * The input data starts with the structure but it might not be appropriately + * aligned. + */ +static HeapTuple +get_changed_tuple(char *change) +{ + HeapTupleData tup_data; + HeapTuple result; + char *src; + + /* + * Ensure alignment before accessing the fields. (This is why we can't use + * heap_copytuple() instead of this function.) + */ + src = change + offsetof(ConcurrentChange, tup_data); + memcpy(&tup_data, src, sizeof(HeapTupleData)); + + result = (HeapTuple) palloc(HEAPTUPLESIZE + tup_data.t_len); + memcpy(result, &tup_data, sizeof(HeapTupleData)); + result->t_data = (HeapTupleHeader) ((char *) result + HEAPTUPLESIZE); + src = change + SizeOfConcurrentChange; + memcpy(result->t_data, src, result->t_len); + + return result; +} + +/* + * Decode logical changes from the WAL sequence up to end_of_wal. + */ +void +repack_decode_concurrent_changes(LogicalDecodingContext *ctx, + XLogRecPtr end_of_wal) +{ + RepackDecodingState *dstate; + ResourceOwner resowner_old; + + /* + * Invalidate the "present" cache before moving to "(recent) history". + */ + InvalidateSystemCaches(); + + dstate = (RepackDecodingState *) ctx->output_writer_private; + resowner_old = CurrentResourceOwner; + CurrentResourceOwner = dstate->resowner; + + PG_TRY(); + { + while (ctx->reader->EndRecPtr < end_of_wal) + { + XLogRecord *record; + XLogSegNo segno_new; + char *errm = NULL; + XLogRecPtr end_lsn; + + record = XLogReadRecord(ctx->reader, &errm); + if (errm) + elog(ERROR, "%s", errm); + + if (record != NULL) + LogicalDecodingProcessRecord(ctx, ctx->reader); + + /* + * If WAL segment boundary has been crossed, inform the decoding + * system that the catalog_xmin can advance. (We can confirm more + * often, but a filling a single WAL segment should not take much + * time.) + */ + end_lsn = ctx->reader->EndRecPtr; + XLByteToSeg(end_lsn, segno_new, wal_segment_size); + if (segno_new != repack_current_segment) + { + LogicalConfirmReceivedLocation(end_lsn); + elog(DEBUG1, "REPACK: confirmed receive location %X/%X", + (uint32) (end_lsn >> 32), (uint32) end_lsn); + repack_current_segment = segno_new; + } + + CHECK_FOR_INTERRUPTS(); + } + InvalidateSystemCaches(); + CurrentResourceOwner = resowner_old; + } + PG_CATCH(); + { + /* clear all timetravel entries */ + InvalidateSystemCaches(); + CurrentResourceOwner = resowner_old; + PG_RE_THROW(); + } + PG_END_TRY(); +} + +/* + * Apply changes that happened during the initial load. + * + * Scan key is passed by caller, so it does not have to be constructed + * multiple times. Key entries have all fields initialized, except for + * sk_argument. + */ +static void +apply_concurrent_changes(RepackDecodingState *dstate, Relation rel, + ScanKey key, int nkeys, IndexInsertState *iistate) +{ + TupleTableSlot *index_slot, + *ident_slot; + HeapTuple tup_old = NULL; + + if (dstate->nchanges == 0) + return; + + /* TupleTableSlot is needed to pass the tuple to ExecInsertIndexTuples(). */ + index_slot = MakeSingleTupleTableSlot(dstate->tupdesc, &TTSOpsHeapTuple); + + /* A slot to fetch tuples from identity index. */ + ident_slot = table_slot_create(rel, NULL); + + while (tuplestore_gettupleslot(dstate->tstore, true, false, + dstate->tsslot)) + { + bool shouldFree; + HeapTuple tup_change, + tup, + tup_exist; + char *change_raw, + *src; + ConcurrentChange change; + bool isnull[1]; + Datum values[1]; + + CHECK_FOR_INTERRUPTS(); + + /* Get the change from the single-column tuple. */ + tup_change = ExecFetchSlotHeapTuple(dstate->tsslot, false, &shouldFree); + heap_deform_tuple(tup_change, dstate->tupdesc_change, values, isnull); + Assert(!isnull[0]); + + /* Make sure we access aligned data. */ + change_raw = (char *) DatumGetByteaP(values[0]); + src = (char *) VARDATA(change_raw); + memcpy(&change, src, SizeOfConcurrentChange); + + /* TRUNCATE change contains no tuple, so process it separately. */ + if (change.kind == CHANGE_TRUNCATE) + { + /* + * All the things that ExecuteTruncateGuts() does (such as firing + * triggers or handling the DROP_CASCADE behavior) should have + * taken place on the source relation. Thus we only do the actual + * truncation of the new relation (and its indexes). + */ + heap_truncate_one_rel(rel); + + pfree(tup_change); + continue; + } + + /* + * Extract the tuple from the change. The tuple is copied here because + * it might be assigned to 'tup_old', in which case it needs to + * survive into the next iteration. + */ + tup = get_changed_tuple(src); + + if (change.kind == CHANGE_UPDATE_OLD) + { + Assert(tup_old == NULL); + tup_old = tup; + } + else if (change.kind == CHANGE_INSERT) + { + Assert(tup_old == NULL); + + apply_concurrent_insert(rel, &change, tup, iistate, index_slot); + + pfree(tup); + } + else if (change.kind == CHANGE_UPDATE_NEW || + change.kind == CHANGE_DELETE) + { + IndexScanDesc ind_scan = NULL; + HeapTuple tup_key; + + if (change.kind == CHANGE_UPDATE_NEW) + { + tup_key = tup_old != NULL ? tup_old : tup; + } + else + { + Assert(tup_old == NULL); + tup_key = tup; + } + + /* + * Find the tuple to be updated or deleted. + */ + tup_exist = find_target_tuple(rel, key, nkeys, tup_key, + iistate, ident_slot, &ind_scan); + if (tup_exist == NULL) + elog(ERROR, "Failed to find target tuple"); + + if (change.kind == CHANGE_UPDATE_NEW) + apply_concurrent_update(rel, tup, tup_exist, &change, iistate, + index_slot); + else + apply_concurrent_delete(rel, tup_exist, &change); + + if (tup_old != NULL) + { + pfree(tup_old); + tup_old = NULL; + } + + pfree(tup); + index_endscan(ind_scan); + } + else + elog(ERROR, "Unrecognized kind of change: %d", change.kind); + + /* + * If a change was applied now, increment CID for next writes and + * update the snapshot so it sees the changes we've applied so far. + */ + if (change.kind != CHANGE_UPDATE_OLD) + { + CommandCounterIncrement(); + UpdateActiveSnapshotCommandId(); + } + + /* TTSOpsMinimalTuple has .get_heap_tuple==NULL. */ + Assert(shouldFree); + pfree(tup_change); + } + + tuplestore_clear(dstate->tstore); + dstate->nchanges = 0; + + /* Cleanup. */ + ExecDropSingleTupleTableSlot(index_slot); + ExecDropSingleTupleTableSlot(ident_slot); +} + +static void +apply_concurrent_insert(Relation rel, ConcurrentChange *change, HeapTuple tup, + IndexInsertState *iistate, TupleTableSlot *index_slot) +{ + List *recheck; + + + /* + * Like simple_heap_insert(), but make sure that the INSERT is not + * logically decoded - see reform_and_rewrite_tuple() for more + * information. + */ + heap_insert(rel, tup, GetCurrentCommandId(true), HEAP_INSERT_NO_LOGICAL, + NULL); + + /* + * Update indexes. + * + * In case functions in the index need the active snapshot and caller + * hasn't set one. + */ + ExecStoreHeapTuple(tup, index_slot, false); + recheck = ExecInsertIndexTuples(iistate->rri, + index_slot, + iistate->estate, + false, /* update */ + false, /* noDupErr */ + NULL, /* specConflict */ + NIL, /* arbiterIndexes */ + false /* onlySummarizing */ + ); + + /* + * If recheck is required, it must have been preformed on the source + * relation by now. (All the logical changes we process here are already + * committed.) + */ + list_free(recheck); + + pgstat_progress_incr_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED, 1); +} + +static void +apply_concurrent_update(Relation rel, HeapTuple tup, HeapTuple tup_target, + ConcurrentChange *change, IndexInsertState *iistate, + TupleTableSlot *index_slot) +{ + LockTupleMode lockmode; + TM_FailureData tmfd; + TU_UpdateIndexes update_indexes; + TM_Result res; + List *recheck; + + /* + * Write the new tuple into the new heap. ('tup' gets the TID assigned + * here.) + * + * Do it like in simple_heap_update(), except for 'wal_logical' (and + * except for 'wait'). + */ + res = heap_update(rel, &tup_target->t_self, tup, + GetCurrentCommandId(true), + InvalidSnapshot, + false, /* no wait - only we are doing changes */ + &tmfd, &lockmode, &update_indexes, + false /* wal_logical */ ); + if (res != TM_Ok) + ereport(ERROR, (errmsg("failed to apply concurrent UPDATE"))); + + ExecStoreHeapTuple(tup, index_slot, false); + + if (update_indexes != TU_None) + { + recheck = ExecInsertIndexTuples(iistate->rri, + index_slot, + iistate->estate, + true, /* update */ + false, /* noDupErr */ + NULL, /* specConflict */ + NIL, /* arbiterIndexes */ + /* onlySummarizing */ + update_indexes == TU_Summarizing); + list_free(recheck); + } + + pgstat_progress_incr_param(PROGRESS_REPACK_HEAP_TUPLES_UPDATED, 1); +} + +static void +apply_concurrent_delete(Relation rel, HeapTuple tup_target, + ConcurrentChange *change) +{ + TM_Result res; + TM_FailureData tmfd; + + /* + * Delete tuple from the new heap. + * + * Do it like in simple_heap_delete(), except for 'wal_logical' (and + * except for 'wait'). + */ + res = heap_delete(rel, &tup_target->t_self, GetCurrentCommandId(true), + InvalidSnapshot, false, + &tmfd, + false, /* no wait - only we are doing changes */ + false /* wal_logical */ ); + + if (res != TM_Ok) + ereport(ERROR, (errmsg("failed to apply concurrent DELETE"))); + + pgstat_progress_incr_param(PROGRESS_REPACK_HEAP_TUPLES_DELETED, 1); +} + +/* + * Find the tuple to be updated or deleted. + * + * 'key' is a pre-initialized scan key, into which the function will put the + * key values. + * + * 'tup_key' is a tuple containing the key values for the scan. + * + * On exit,'*scan_p' contains the scan descriptor used. The caller must close + * it when he no longer needs the tuple returned. + */ +static HeapTuple +find_target_tuple(Relation rel, ScanKey key, int nkeys, HeapTuple tup_key, + IndexInsertState *iistate, + TupleTableSlot *ident_slot, IndexScanDesc *scan_p) +{ + IndexScanDesc scan; + Form_pg_index ident_form; + int2vector *ident_indkey; + HeapTuple result = NULL; + + /* XXX no instrumentation for now */ + scan = index_beginscan(rel, iistate->ident_index, GetActiveSnapshot(), + NULL, nkeys, 0); + *scan_p = scan; + index_rescan(scan, key, nkeys, NULL, 0); + + /* Info needed to retrieve key values from heap tuple. */ + ident_form = iistate->ident_index->rd_index; + ident_indkey = &ident_form->indkey; + + /* Use the incoming tuple to finalize the scan key. */ + for (int i = 0; i < scan->numberOfKeys; i++) + { + ScanKey entry; + bool isnull; + int16 attno_heap; + + entry = &scan->keyData[i]; + attno_heap = ident_indkey->values[i]; + entry->sk_argument = heap_getattr(tup_key, + attno_heap, + rel->rd_att, + &isnull); + Assert(!isnull); + } + if (index_getnext_slot(scan, ForwardScanDirection, ident_slot)) + { + bool shouldFree; + + result = ExecFetchSlotHeapTuple(ident_slot, false, &shouldFree); + /* TTSOpsBufferHeapTuple has .get_heap_tuple != NULL. */ + Assert(!shouldFree); + } + + return result; +} + +/* + * Decode and apply concurrent changes. + * + * Pass rel_src iff its reltoastrelid is needed. + */ +static void +process_concurrent_changes(LogicalDecodingContext *ctx, XLogRecPtr end_of_wal, + Relation rel_dst, Relation rel_src, ScanKey ident_key, + int ident_key_nentries, IndexInsertState *iistate) +{ + RepackDecodingState *dstate; + + pgstat_progress_update_param(PROGRESS_REPACK_PHASE, + PROGRESS_REPACK_PHASE_CATCH_UP); + + dstate = (RepackDecodingState *) ctx->output_writer_private; + + repack_decode_concurrent_changes(ctx, end_of_wal); + + if (dstate->nchanges == 0) + return; + + PG_TRY(); + { + /* + * Make sure that TOAST values can eventually be accessed via the old + * relation - see comment in copy_table_data(). + */ + if (rel_src) + rel_dst->rd_toastoid = rel_src->rd_rel->reltoastrelid; + + apply_concurrent_changes(dstate, rel_dst, ident_key, + ident_key_nentries, iistate); + } + PG_FINALLY(); + { + if (rel_src) + rel_dst->rd_toastoid = InvalidOid; + } + PG_END_TRY(); +} + +static IndexInsertState * +get_index_insert_state(Relation relation, Oid ident_index_id) +{ + EState *estate; + int i; + IndexInsertState *result; + + result = (IndexInsertState *) palloc0(sizeof(IndexInsertState)); + estate = CreateExecutorState(); + + result->rri = (ResultRelInfo *) palloc(sizeof(ResultRelInfo)); + InitResultRelInfo(result->rri, relation, 0, 0, 0); + ExecOpenIndices(result->rri, false); + + /* + * Find the relcache entry of the identity index so that we spend no extra + * effort to open / close it. + */ + for (i = 0; i < result->rri->ri_NumIndices; i++) + { + Relation ind_rel; + + ind_rel = result->rri->ri_IndexRelationDescs[i]; + if (ind_rel->rd_id == ident_index_id) + result->ident_index = ind_rel; + } + if (result->ident_index == NULL) + elog(ERROR, "Failed to open identity index"); + + /* Only initialize fields needed by ExecInsertIndexTuples(). */ + result->estate = estate; + + return result; +} + +/* + * Build scan key to process logical changes. + */ +static ScanKey +build_identity_key(Oid ident_idx_oid, Relation rel_src, int *nentries) +{ + Relation ident_idx_rel; + Form_pg_index ident_idx; + int n, + i; + ScanKey result; + + Assert(OidIsValid(ident_idx_oid)); + ident_idx_rel = index_open(ident_idx_oid, AccessShareLock); + ident_idx = ident_idx_rel->rd_index; + n = ident_idx->indnatts; + result = (ScanKey) palloc(sizeof(ScanKeyData) * n); + for (i = 0; i < n; i++) + { + ScanKey entry; + int16 relattno; + Form_pg_attribute att; + Oid opfamily, + opcintype, + opno, + opcode; + + entry = &result[i]; + relattno = ident_idx->indkey.values[i]; + if (relattno >= 1) + { + TupleDesc desc; + + desc = rel_src->rd_att; + att = TupleDescAttr(desc, relattno - 1); + } + else + elog(ERROR, "Unexpected attribute number %d in index", relattno); + + opfamily = ident_idx_rel->rd_opfamily[i]; + opcintype = ident_idx_rel->rd_opcintype[i]; + opno = get_opfamily_member(opfamily, opcintype, opcintype, + BTEqualStrategyNumber); + + if (!OidIsValid(opno)) + elog(ERROR, "Failed to find = operator for type %u", opcintype); + + opcode = get_opcode(opno); + if (!OidIsValid(opcode)) + elog(ERROR, "Failed to find = operator for operator %u", opno); + + /* Initialize everything but argument. */ + ScanKeyInit(entry, + i + 1, + BTEqualStrategyNumber, opcode, + (Datum) NULL); + entry->sk_collation = att->attcollation; + } + index_close(ident_idx_rel, AccessShareLock); + + *nentries = n; + return result; +} + +static void +free_index_insert_state(IndexInsertState *iistate) +{ + ExecCloseIndices(iistate->rri); + FreeExecutorState(iistate->estate); + pfree(iistate->rri); + pfree(iistate); +} + +static void +cleanup_logical_decoding(LogicalDecodingContext *ctx) +{ + RepackDecodingState *dstate; + + dstate = (RepackDecodingState *) ctx->output_writer_private; + + ExecDropSingleTupleTableSlot(dstate->tsslot); + FreeTupleDesc(dstate->tupdesc_change); + FreeTupleDesc(dstate->tupdesc); + tuplestore_end(dstate->tstore); + + FreeDecodingContext(ctx); +} + +/* + * The final steps of rebuild_relation() for concurrent processing. + * + * On entry, NewHeap is locked in AccessExclusiveLock mode. OldHeap and its + * clustering index (if one is passed) are still locked in a mode that allows + * concurrent data changes. On exit, both tables and their indexes are closed, + * but locked in AccessExclusiveLock mode. + */ +static void +rebuild_relation_finish_concurrent(Relation NewHeap, Relation OldHeap, + Relation cl_index, + LogicalDecodingContext *ctx, + bool swap_toast_by_content, + TransactionId frozenXid, + MultiXactId cutoffMulti) +{ + LOCKMODE lockmode_old PG_USED_FOR_ASSERTS_ONLY; + List *ind_oids_new; + Oid old_table_oid = RelationGetRelid(OldHeap); + Oid new_table_oid = RelationGetRelid(NewHeap); + List *ind_oids_old = RelationGetIndexList(OldHeap); + ListCell *lc, + *lc2; + char relpersistence; + bool is_system_catalog; + Oid ident_idx_old, + ident_idx_new; + IndexInsertState *iistate; + ScanKey ident_key; + int ident_key_nentries; + XLogRecPtr wal_insert_ptr, + end_of_wal; + char dummy_rec_data = '\0'; + Relation *ind_refs, + *ind_refs_p; + int nind; + + /* Like in cluster_rel(). */ + lockmode_old = ShareUpdateExclusiveLock; + Assert(CheckRelationLockedByMe(OldHeap, lockmode_old, false)); + Assert(cl_index == NULL || + CheckRelationLockedByMe(cl_index, lockmode_old, false)); + /* This is expected from the caller. */ + Assert(CheckRelationLockedByMe(NewHeap, AccessExclusiveLock, false)); + + ident_idx_old = RelationGetReplicaIndex(OldHeap); + + /* + * Unlike the exclusive case, we build new indexes for the new relation + * rather than swapping the storage and reindexing the old relation. The + * point is that the index build can take some time, so we do it before we + * get AccessExclusiveLock on the old heap and therefore we cannot swap + * the heap storage yet. + * + * index_create() will lock the new indexes using AccessExclusiveLock - no + * need to change that. + * + * We assume that ShareUpdateExclusiveLock on the table prevents anyone + * from dropping the existing indexes or adding new ones, so the lists of + * old and new indexes should match at the swap time. On the other hand we + * do not block ALTER INDEX commands that do not require table lock (e.g. + * ALTER INDEX ... SET ...). + * + * XXX Should we check a the end of our work if another transaction + * executed such a command and issue a NOTICE that we might have discarded + * its effects? (For example, someone changes storage parameter after we + * have created the new index, the new value of that parameter is lost.) + * Alternatively, we can lock all the indexes now in a mode that blocks + * all the ALTER INDEX commands (ShareUpdateExclusiveLock ?), and keep + * them locked till the end of the transactions. That might increase the + * risk of deadlock during the lock upgrade below, however SELECT / DML + * queries should not be involved in such a deadlock. + */ + ind_oids_new = build_new_indexes(NewHeap, OldHeap, ind_oids_old); + + /* + * Processing shouldn't start w/o valid identity index. + */ + Assert(OidIsValid(ident_idx_old)); + + /* Find "identity index" on the new relation. */ + ident_idx_new = InvalidOid; + forboth(lc, ind_oids_old, lc2, ind_oids_new) + { + Oid ind_old = lfirst_oid(lc); + Oid ind_new = lfirst_oid(lc2); + + if (ident_idx_old == ind_old) + { + ident_idx_new = ind_new; + break; + } + } + if (!OidIsValid(ident_idx_new)) + + /* + * Should not happen, given our lock on the old relation. + */ + ereport(ERROR, + (errmsg("Identity index missing on the new relation"))); + + /* Executor state to update indexes. */ + iistate = get_index_insert_state(NewHeap, ident_idx_new); + + /* + * Build scan key that we'll use to look for rows to be updated / deleted + * during logical decoding. + */ + ident_key = build_identity_key(ident_idx_new, OldHeap, &ident_key_nentries); + + /* + * During testing, wait for another backend to perform concurrent data + * changes which we will process below. + */ + INJECTION_POINT("repack-concurrently-before-lock", NULL); + + /* + * Flush all WAL records inserted so far (possibly except for the last + * incomplete page, see GetInsertRecPtr), to minimize the amount of data + * we need to flush while holding exclusive lock on the source table. + */ + wal_insert_ptr = GetInsertRecPtr(); + XLogFlush(wal_insert_ptr); + end_of_wal = GetFlushRecPtr(NULL); + + /* + * Apply concurrent changes first time, to minimize the time we need to + * hold AccessExclusiveLock. (Quite some amount of WAL could have been + * written during the data copying and index creation.) + */ + process_concurrent_changes(ctx, end_of_wal, NewHeap, + swap_toast_by_content ? OldHeap : NULL, + ident_key, ident_key_nentries, iistate); + + /* + * Acquire AccessExclusiveLock on the table, its TOAST relation (if there + * is one), all its indexes, so that we can swap the files. + * + * Before that, unlock the index temporarily to avoid deadlock in case + * another transaction is trying to lock it while holding the lock on the + * table. + */ + if (cl_index) + { + index_close(cl_index, ShareUpdateExclusiveLock); + cl_index = NULL; + } + /* For the same reason, unlock TOAST relation. */ + if (OldHeap->rd_rel->reltoastrelid) + LockRelationOid(OldHeap->rd_rel->reltoastrelid, AccessExclusiveLock); + /* Finally lock the table */ + LockRelationOid(old_table_oid, AccessExclusiveLock); + + /* + * Lock all indexes now, not only the clustering one: all indexes need to + * have their files swapped. While doing that, store their relation + * references in an array, to handle predicate locks below. + */ + ind_refs_p = ind_refs = palloc_array(Relation, list_length(ind_oids_old)); + nind = 0; + foreach(lc, ind_oids_old) + { + Oid ind_oid; + Relation index; + + ind_oid = lfirst_oid(lc); + index = index_open(ind_oid, AccessExclusiveLock); + + /* + * TODO 1) Do we need to check if ALTER INDEX was executed since the + * new index was created in build_new_indexes()? 2) Specifically for + * the clustering index, should check_index_is_clusterable() be called + * here? (Not sure about the latter: ShareUpdateExclusiveLock on the + * table probably blocks all commands that affect the result of + * check_index_is_clusterable().) + */ + *ind_refs_p = index; + ind_refs_p++; + nind++; + } + + /* + * In addition, lock the OldHeap's TOAST relation exclusively - again, the + * lock is needed to swap the files. + */ + if (OidIsValid(OldHeap->rd_rel->reltoastrelid)) + LockRelationOid(OldHeap->rd_rel->reltoastrelid, AccessExclusiveLock); + + /* + * Tuples and pages of the old heap will be gone, but the heap will stay. + */ + TransferPredicateLocksToHeapRelation(OldHeap); + /* The same for indexes. */ + for (int i = 0; i < nind; i++) + { + Relation index = ind_refs[i]; + + TransferPredicateLocksToHeapRelation(index); + + /* + * References to indexes on the old relation are not needed anymore, + * however locks stay till the end of the transaction. + */ + index_close(index, NoLock); + } + pfree(ind_refs); + + /* + * Flush anything we see in WAL, to make sure that all changes committed + * while we were waiting for the exclusive lock are available for + * decoding. This should not be necessary if all backends had + * synchronous_commit set, but we can't rely on this setting. + * + * Unfortunately, GetInsertRecPtr() may lag behind the actual insert + * position, and GetLastImportantRecPtr() points at the start of the last + * record rather than at the end. Thus the simplest way to determine the + * insert position is to insert a dummy record and use its LSN. + * + * XXX Consider using GetLastImportantRecPtr() and adding the size of the + * last record (plus the total size of all the page headers the record + * spans)? + */ + XLogBeginInsert(); + XLogRegisterData(&dummy_rec_data, 1); + wal_insert_ptr = XLogInsert(RM_XLOG_ID, XLOG_NOOP); + XLogFlush(wal_insert_ptr); + end_of_wal = GetFlushRecPtr(NULL); + + /* Apply the concurrent changes again. */ + process_concurrent_changes(ctx, end_of_wal, NewHeap, + swap_toast_by_content ? OldHeap : NULL, + ident_key, ident_key_nentries, iistate); + + /* Remember info about rel before closing OldHeap */ + relpersistence = OldHeap->rd_rel->relpersistence; + is_system_catalog = IsSystemRelation(OldHeap); + + pgstat_progress_update_param(PROGRESS_REPACK_PHASE, + PROGRESS_REPACK_PHASE_SWAP_REL_FILES); + + /* + * Even ShareUpdateExclusiveLock should have prevented others from + * creating / dropping indexes (even using the CONCURRENTLY option), so we + * do not need to check whether the lists match. + */ + forboth(lc, ind_oids_old, lc2, ind_oids_new) + { + Oid ind_old = lfirst_oid(lc); + Oid ind_new = lfirst_oid(lc2); + Oid mapped_tables[4]; + + /* Zero out possible results from swapped_relation_files */ + memset(mapped_tables, 0, sizeof(mapped_tables)); + + swap_relation_files(ind_old, ind_new, + (old_table_oid == RelationRelationId), + swap_toast_by_content, + true, + InvalidTransactionId, + InvalidMultiXactId, + mapped_tables); + +#ifdef USE_ASSERT_CHECKING + + /* + * Concurrent processing is not supported for system relations, so + * there should be no mapped tables. + */ + for (int i = 0; i < 4; i++) + Assert(mapped_tables[i] == 0); +#endif + } + + /* The new indexes must be visible for deletion. */ + CommandCounterIncrement(); + + /* Close the old heap but keep lock until transaction commit. */ + table_close(OldHeap, NoLock); + /* Close the new heap. (We didn't have to open its indexes). */ + table_close(NewHeap, NoLock); + + /* Cleanup what we don't need anymore. (And close the identity index.) */ + pfree(ident_key); + free_index_insert_state(iistate); + + /* + * Swap the relations and their TOAST relations and TOAST indexes. This + * also drops the new relation and its indexes. + * + * (System catalogs are currently not supported.) + */ + Assert(!is_system_catalog); + finish_heap_swap(old_table_oid, new_table_oid, + is_system_catalog, + swap_toast_by_content, + false, true, false, + frozenXid, cutoffMulti, + relpersistence); +} + +/* + * Build indexes on NewHeap according to those on OldHeap. + * + * OldIndexes is the list of index OIDs on OldHeap. + * + * A list of OIDs of the corresponding indexes created on NewHeap is + * returned. The order of items does match, so we can use these arrays to swap + * index storage. + */ +static List * +build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes) +{ + ListCell *lc; + List *result = NIL; + + pgstat_progress_update_param(PROGRESS_REPACK_PHASE, + PROGRESS_REPACK_PHASE_REBUILD_INDEX); + + foreach(lc, OldIndexes) + { + Oid ind_oid, + ind_oid_new; + char *newName; + Relation ind; + + ind_oid = lfirst_oid(lc); + ind = index_open(ind_oid, AccessShareLock); + + newName = ChooseRelationName(get_rel_name(ind_oid), + NULL, + "repacknew", + get_rel_namespace(ind->rd_index->indrelid), + false); + ind_oid_new = index_create_copy(NewHeap, ind_oid, + ind->rd_rel->reltablespace, newName, + false); + result = lappend_oid(result, ind_oid_new); + + index_close(ind, AccessShareLock); + } + + return result; +} diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c index 188e26f0e6e..71b73c21ebf 100644 --- a/src/backend/commands/matview.c +++ b/src/backend/commands/matview.c @@ -904,7 +904,7 @@ refresh_by_match_merge(Oid matviewOid, Oid tempOid, Oid relowner, static void refresh_by_heap_swap(Oid matviewOid, Oid OIDNewHeap, char relpersistence) { - finish_heap_swap(matviewOid, OIDNewHeap, false, false, true, true, + finish_heap_swap(matviewOid, OIDNewHeap, false, false, true, true, true, RecentXmin, ReadNextMultiXactId(), relpersistence); } diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c6dd2e020da..d83136a2657 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -5989,6 +5989,7 @@ ATRewriteTables(AlterTableStmt *parsetree, List **wqueue, LOCKMODE lockmode, finish_heap_swap(tab->relid, OIDNewHeap, false, false, true, !OidIsValid(tab->newTableSpace), + true, RecentXmin, ReadNextMultiXactId(), persistence); diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index 8863ad0e8bd..6de9d0ba39d 100644 --- a/src/backend/commands/vacuum.c +++ b/src/backend/commands/vacuum.c @@ -125,7 +125,7 @@ static void vac_truncate_clog(TransactionId frozenXID, TransactionId lastSaneFrozenXid, MultiXactId lastSaneMinMulti); static bool vacuum_rel(Oid relid, RangeVar *relation, VacuumParams params, - BufferAccessStrategy bstrategy); + BufferAccessStrategy bstrategy, bool isTopLevel); static double compute_parallel_delay(void); static VacOptValue get_vacoptval_from_boolean(DefElem *def); static bool vac_tid_reaped(ItemPointer itemptr, void *state); @@ -633,7 +633,8 @@ vacuum(List *relations, const VacuumParams params, BufferAccessStrategy bstrateg if (params.options & VACOPT_VACUUM) { - if (!vacuum_rel(vrel->oid, vrel->relation, params, bstrategy)) + if (!vacuum_rel(vrel->oid, vrel->relation, params, bstrategy, + isTopLevel)) continue; } @@ -1997,7 +1998,7 @@ vac_truncate_clog(TransactionId frozenXID, */ static bool vacuum_rel(Oid relid, RangeVar *relation, VacuumParams params, - BufferAccessStrategy bstrategy) + BufferAccessStrategy bstrategy, bool isTopLevel) { LOCKMODE lmode; Relation rel; @@ -2288,7 +2289,7 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams params, /* VACUUM FULL is now a variant of CLUSTER; see cluster.c */ cluster_rel(REPACK_COMMAND_VACUUMFULL, false, rel, InvalidOid, - &cluster_params); + &cluster_params, isTopLevel); /* cluster_rel closes the relation, but keeps lock */ rel = NULL; @@ -2331,7 +2332,8 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams params, toast_vacuum_params.options |= VACOPT_PROCESS_MAIN; toast_vacuum_params.toast_parent = relid; - vacuum_rel(toast_relid, NULL, toast_vacuum_params, bstrategy); + vacuum_rel(toast_relid, NULL, toast_vacuum_params, bstrategy, + isTopLevel); } /* diff --git a/src/backend/meson.build b/src/backend/meson.build index 2b0db214804..50aa385a581 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -194,5 +194,6 @@ pg_test_mod_args = pg_mod_args + { subdir('jit/llvm') subdir('replication/libpqwalreceiver') subdir('replication/pgoutput') +subdir('replication/pgoutput_repack') subdir('snowball') subdir('utils/mb/conversion_procs') diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index cc03f0706e9..5dc4ae58ffe 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -33,6 +33,7 @@ #include "access/xlogreader.h" #include "access/xlogrecord.h" #include "catalog/pg_control.h" +#include "commands/cluster.h" #include "replication/decode.h" #include "replication/logical.h" #include "replication/message.h" @@ -472,6 +473,88 @@ heap_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) TransactionId xid = XLogRecGetXid(buf->record); SnapBuild *builder = ctx->snapshot_builder; + /* + * If the change is not intended for logical decoding, do not even + * establish transaction for it - REPACK CONCURRENTLY is the typical use + * case. + * + * First, check if REPACK CONCURRENTLY is being performed by this backend. + * If so, only decode data changes of the table that it is processing, and + * the changes of its TOAST relation. + * + * (TOAST locator should not be set unless the main is.) + */ + Assert(!OidIsValid(repacked_rel_toast_locator.relNumber) || + OidIsValid(repacked_rel_locator.relNumber)); + + if (OidIsValid(repacked_rel_locator.relNumber)) + { + XLogReaderState *r = buf->record; + RelFileLocator locator; + + /* Not all records contain the block. */ + if (XLogRecGetBlockTagExtended(r, 0, &locator, NULL, NULL, NULL) && + !RelFileLocatorEquals(locator, repacked_rel_locator) && + (!OidIsValid(repacked_rel_toast_locator.relNumber) || + !RelFileLocatorEquals(locator, repacked_rel_toast_locator))) + return; + } + + /* + * Second, skip records which do not contain sufficient information for + * the decoding. + * + * The problem we solve here is that REPACK CONCURRENTLY generates WAL + * when doing changes in the new table. Those changes should not be useful + * for any other user (such as logical replication subscription) because + * the new table will eventually be dropped (after REPACK CONCURRENTLY has + * assigned its file to the "old table"). + */ + switch (info) + { + case XLOG_HEAP_INSERT: + { + xl_heap_insert *rec; + + rec = (xl_heap_insert *) XLogRecGetData(buf->record); + + /* + * This does happen when 1) raw_heap_insert marks the TOAST + * record as HEAP_INSERT_NO_LOGICAL, 2) REPACK CONCURRENTLY + * replays inserts performed by other backends. + */ + if ((rec->flags & XLH_INSERT_CONTAINS_NEW_TUPLE) == 0) + return; + + break; + } + + case XLOG_HEAP_HOT_UPDATE: + case XLOG_HEAP_UPDATE: + { + xl_heap_update *rec; + + rec = (xl_heap_update *) XLogRecGetData(buf->record); + if ((rec->flags & + (XLH_UPDATE_CONTAINS_NEW_TUPLE | + XLH_UPDATE_CONTAINS_OLD_TUPLE | + XLH_UPDATE_CONTAINS_OLD_KEY)) == 0) + return; + + break; + } + + case XLOG_HEAP_DELETE: + { + xl_heap_delete *rec; + + rec = (xl_heap_delete *) XLogRecGetData(buf->record); + if (rec->flags & XLH_DELETE_NO_LOGICAL) + return; + break; + } + } + ReorderBufferProcessXid(ctx->reorder, xid, buf->origptr); /* diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 6eaa40f6acf..56ea521e8c6 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -486,6 +486,26 @@ SnapBuildInitialSnapshot(SnapBuild *builder) return SnapBuildMVCCFromHistoric(snap, true); } +/* + * Build an MVCC snapshot for the initial data load performed by REPACK + * CONCURRENTLY command. + * + * The snapshot will only be used to scan one particular relation, which is + * treated like a catalog (therefore ->building_full_snapshot is not + * important), and the caller should already have a replication slot setup (so + * we do not set MyProc->xmin). XXX Do we yet need to add some restrictions? + */ +Snapshot +SnapBuildInitialSnapshotForRepack(SnapBuild *builder) +{ + Snapshot snap; + + Assert(builder->state == SNAPBUILD_CONSISTENT); + + snap = SnapBuildBuildSnapshot(builder); + return SnapBuildMVCCFromHistoric(snap, false); +} + /* * Turn a historic MVCC snapshot into an ordinary MVCC snapshot. * diff --git a/src/backend/replication/pgoutput_repack/Makefile b/src/backend/replication/pgoutput_repack/Makefile new file mode 100644 index 00000000000..4efeb713b70 --- /dev/null +++ b/src/backend/replication/pgoutput_repack/Makefile @@ -0,0 +1,32 @@ +#------------------------------------------------------------------------- +# +# Makefile-- +# Makefile for src/backend/replication/pgoutput_repack +# +# IDENTIFICATION +# src/backend/replication/pgoutput_repack +# +#------------------------------------------------------------------------- + +subdir = src/backend/replication/pgoutput_repack +top_builddir = ../../../.. +include $(top_builddir)/src/Makefile.global + +OBJS = \ + $(WIN32RES) \ + pgoutput_repack.o +PGFILEDESC = "pgoutput_repack - logical replication output plugin for REPACK command" +NAME = pgoutput_repack + +all: all-shared-lib + +include $(top_srcdir)/src/Makefile.shlib + +install: all installdirs install-lib + +installdirs: installdirs-lib + +uninstall: uninstall-lib + +clean distclean: clean-lib + rm -f $(OBJS) diff --git a/src/backend/replication/pgoutput_repack/meson.build b/src/backend/replication/pgoutput_repack/meson.build new file mode 100644 index 00000000000..133e865a4a0 --- /dev/null +++ b/src/backend/replication/pgoutput_repack/meson.build @@ -0,0 +1,18 @@ +# Copyright (c) 2022-2024, PostgreSQL Global Development Group + +pgoutput_repack_sources = files( + 'pgoutput_repack.c', +) + +if host_system == 'windows' + pgoutput_repack_sources += rc_lib_gen.process(win32ver_rc, extra_args: [ + '--NAME', 'pgoutput_repack', + '--FILEDESC', 'pgoutput_repack - logical replication output plugin for REPACK command',]) +endif + +pgoutput_repack = shared_module('pgoutput_repack', + pgoutput_repack_sources, + kwargs: pg_mod_args, +) + +backend_targets += pgoutput_repack diff --git a/src/backend/replication/pgoutput_repack/pgoutput_repack.c b/src/backend/replication/pgoutput_repack/pgoutput_repack.c new file mode 100644 index 00000000000..687fbbc59bb --- /dev/null +++ b/src/backend/replication/pgoutput_repack/pgoutput_repack.c @@ -0,0 +1,288 @@ +/*------------------------------------------------------------------------- + * + * pgoutput_cluster.c + * Logical Replication output plugin for REPACK command + * + * Copyright (c) 2012-2024, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/backend/replication/pgoutput_cluster/pgoutput_cluster.c + * + *------------------------------------------------------------------------- + */ +#include "postgres.h" + +#include "access/heaptoast.h" +#include "commands/cluster.h" +#include "replication/snapbuild.h" + +PG_MODULE_MAGIC; + +static void plugin_startup(LogicalDecodingContext *ctx, + OutputPluginOptions *opt, bool is_init); +static void plugin_shutdown(LogicalDecodingContext *ctx); +static void plugin_begin_txn(LogicalDecodingContext *ctx, + ReorderBufferTXN *txn); +static void plugin_commit_txn(LogicalDecodingContext *ctx, + ReorderBufferTXN *txn, XLogRecPtr commit_lsn); +static void plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn, + Relation rel, ReorderBufferChange *change); +static void plugin_truncate(struct LogicalDecodingContext *ctx, + ReorderBufferTXN *txn, int nrelations, + Relation relations[], + ReorderBufferChange *change); +static void store_change(LogicalDecodingContext *ctx, + ConcurrentChangeKind kind, HeapTuple tuple); + +void +_PG_output_plugin_init(OutputPluginCallbacks *cb) +{ + AssertVariableIsOfType(&_PG_output_plugin_init, LogicalOutputPluginInit); + + cb->startup_cb = plugin_startup; + cb->begin_cb = plugin_begin_txn; + cb->change_cb = plugin_change; + cb->truncate_cb = plugin_truncate; + cb->commit_cb = plugin_commit_txn; + cb->shutdown_cb = plugin_shutdown; +} + + +/* initialize this plugin */ +static void +plugin_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt, + bool is_init) +{ + ctx->output_plugin_private = NULL; + + /* Probably unnecessary, as we don't use the SQL interface ... */ + opt->output_type = OUTPUT_PLUGIN_BINARY_OUTPUT; + + if (ctx->output_plugin_options != NIL) + { + ereport(ERROR, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("This plugin does not expect any options"))); + } +} + +static void +plugin_shutdown(LogicalDecodingContext *ctx) +{ +} + +/* + * As we don't release the slot during processing of particular table, there's + * no room for SQL interface, even for debugging purposes. Therefore we need + * neither OutputPluginPrepareWrite() nor OutputPluginWrite() in the plugin + * callbacks. (Although we might want to write custom callbacks, this API + * seems to be unnecessarily generic for our purposes.) + */ + +/* BEGIN callback */ +static void +plugin_begin_txn(LogicalDecodingContext *ctx, ReorderBufferTXN *txn) +{ +} + +/* COMMIT callback */ +static void +plugin_commit_txn(LogicalDecodingContext *ctx, ReorderBufferTXN *txn, + XLogRecPtr commit_lsn) +{ +} + +/* + * Callback for individual changed tuples + */ +static void +plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn, + Relation relation, ReorderBufferChange *change) +{ + RepackDecodingState *dstate; + + dstate = (RepackDecodingState *) ctx->output_writer_private; + + /* Only interested in one particular relation. */ + if (relation->rd_id != dstate->relid) + return; + + /* Decode entry depending on its type */ + switch (change->action) + { + case REORDER_BUFFER_CHANGE_INSERT: + { + HeapTuple newtuple; + + newtuple = change->data.tp.newtuple != NULL ? + change->data.tp.newtuple : NULL; + + /* + * Identity checks in the main function should have made this + * impossible. + */ + if (newtuple == NULL) + elog(ERROR, "Incomplete insert info."); + + store_change(ctx, CHANGE_INSERT, newtuple); + } + break; + case REORDER_BUFFER_CHANGE_UPDATE: + { + HeapTuple oldtuple, + newtuple; + + oldtuple = change->data.tp.oldtuple != NULL ? + change->data.tp.oldtuple : NULL; + newtuple = change->data.tp.newtuple != NULL ? + change->data.tp.newtuple : NULL; + + if (newtuple == NULL) + elog(ERROR, "Incomplete update info."); + + if (oldtuple != NULL) + store_change(ctx, CHANGE_UPDATE_OLD, oldtuple); + + store_change(ctx, CHANGE_UPDATE_NEW, newtuple); + } + break; + case REORDER_BUFFER_CHANGE_DELETE: + { + HeapTuple oldtuple; + + oldtuple = change->data.tp.oldtuple ? + change->data.tp.oldtuple : NULL; + + if (oldtuple == NULL) + elog(ERROR, "Incomplete delete info."); + + store_change(ctx, CHANGE_DELETE, oldtuple); + } + break; + default: + /* Should not come here */ + Assert(false); + break; + } +} + +static void +plugin_truncate(struct LogicalDecodingContext *ctx, ReorderBufferTXN *txn, + int nrelations, Relation relations[], + ReorderBufferChange *change) +{ + RepackDecodingState *dstate; + int i; + Relation relation = NULL; + + dstate = (RepackDecodingState *) ctx->output_writer_private; + + /* Find the relation we are processing. */ + for (i = 0; i < nrelations; i++) + { + relation = relations[i]; + + if (RelationGetRelid(relation) == dstate->relid) + break; + } + + /* Is this truncation of another relation? */ + if (i == nrelations) + return; + + store_change(ctx, CHANGE_TRUNCATE, NULL); +} + +/* Store concurrent data change. */ +static void +store_change(LogicalDecodingContext *ctx, ConcurrentChangeKind kind, + HeapTuple tuple) +{ + RepackDecodingState *dstate; + char *change_raw; + ConcurrentChange change; + bool flattened = false; + Size size; + Datum values[1]; + bool isnull[1]; + char *dst, + *dst_start; + + dstate = (RepackDecodingState *) ctx->output_writer_private; + + size = MAXALIGN(VARHDRSZ) + SizeOfConcurrentChange; + + if (tuple) + { + /* + * ReorderBufferCommit() stores the TOAST chunks in its private memory + * context and frees them after having called apply_change(). + * Therefore we need flat copy (including TOAST) that we eventually + * copy into the memory context which is available to + * decode_concurrent_changes(). + */ + if (HeapTupleHasExternal(tuple)) + { + /* + * toast_flatten_tuple_to_datum() might be more convenient but we + * don't want the decompression it does. + */ + tuple = toast_flatten_tuple(tuple, dstate->tupdesc); + flattened = true; + } + + size += tuple->t_len; + } + + /* XXX Isn't there any function / macro to do this? */ + if (size >= 0x3FFFFFFF) + elog(ERROR, "Change is too big."); + + /* Construct the change. */ + change_raw = (char *) palloc0(size); + SET_VARSIZE(change_raw, size); + + /* + * Since the varlena alignment might not be sufficient for the structure, + * set the fields in a local instance and remember where it should + * eventually be copied. + */ + change.kind = kind; + dst_start = (char *) VARDATA(change_raw); + + /* No other information is needed for TRUNCATE. */ + if (change.kind == CHANGE_TRUNCATE) + { + memcpy(dst_start, &change, SizeOfConcurrentChange); + goto store; + } + + /* + * Copy the tuple. + * + * CAUTION: change->tup_data.t_data must be fixed on retrieval! + */ + memcpy(&change.tup_data, tuple, sizeof(HeapTupleData)); + dst = dst_start + SizeOfConcurrentChange; + memcpy(dst, tuple->t_data, tuple->t_len); + + /* The data has been copied. */ + if (flattened) + pfree(tuple); + +store: + /* Copy the structure so it can be stored. */ + memcpy(dst_start, &change, SizeOfConcurrentChange); + + /* Store as tuple of 1 bytea column. */ + values[0] = PointerGetDatum(change_raw); + isnull[0] = false; + tuplestore_putvalues(dstate->tstore, dstate->tupdesc_change, + values, isnull); + + /* Accounting. */ + dstate->nchanges++; + + /* Cleanup. */ + pfree(change_raw); +} diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c index 2fa045e6b0f..e9ddf39500c 100644 --- a/src/backend/storage/ipc/ipci.c +++ b/src/backend/storage/ipc/ipci.c @@ -25,6 +25,7 @@ #include "access/xlogprefetcher.h" #include "access/xlogrecovery.h" #include "commands/async.h" +#include "commands/cluster.h" #include "miscadmin.h" #include "pgstat.h" #include "postmaster/autovacuum.h" diff --git a/src/backend/storage/lmgr/generate-lwlocknames.pl b/src/backend/storage/lmgr/generate-lwlocknames.pl index cd3e43c448a..519f3953638 100644 --- a/src/backend/storage/lmgr/generate-lwlocknames.pl +++ b/src/backend/storage/lmgr/generate-lwlocknames.pl @@ -162,7 +162,7 @@ while (<$lwlocklist>) die "$wait_event_lwlocks[$lwlock_count] defined in wait_event_names.txt but " - . " missing from lwlocklist.h" + . "missing from lwlocklist.h" if $lwlock_count < scalar @wait_event_lwlocks; die diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index 6fe268a8eec..d27a4c30548 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -64,6 +64,7 @@ #include "catalog/pg_type.h" #include "catalog/schemapg.h" #include "catalog/storage.h" +#include "commands/cluster.h" #include "commands/policy.h" #include "commands/publicationcmds.h" #include "commands/trigger.h" diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 70a6b8902d1..7f1c220e00b 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -213,7 +213,6 @@ static List *exportedSnapshots = NIL; /* Prototypes for local functions */ static void UnregisterSnapshotNoOwner(Snapshot snapshot); -static void FreeSnapshot(Snapshot snapshot); static void SnapshotResetXmin(void); /* ResourceOwner callbacks to track snapshot references */ @@ -646,7 +645,7 @@ CopySnapshot(Snapshot snapshot) * FreeSnapshot * Free the memory associated with a snapshot. */ -static void +void FreeSnapshot(Snapshot snapshot) { Assert(snapshot->regd_count == 0); diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 31c1ce5dc09..f23efa12d69 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -4998,18 +4998,27 @@ match_previous_words(int pattern_id, } /* REPACK */ - else if (Matches("REPACK")) + else if (Matches("REPACK") || Matches("REPACK", "(*)")) + COMPLETE_WITH_SCHEMA_QUERY_PLUS(Query_for_list_of_clusterables, + "CONCURRENTLY"); + else if (Matches("REPACK", "CONCURRENTLY")) COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_clusterables); - else if (Matches("REPACK", "(*)")) + else if (Matches("REPACK", "(*)", "CONCURRENTLY")) COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_clusterables); - /* If we have REPACK <sth>, then add "USING INDEX" */ - else if (Matches("REPACK", MatchAnyExcept("("))) + /* If we have REPACK [ CONCURRENTLY ] <sth>, then add "USING INDEX" */ + else if (Matches("REPACK", MatchAnyExcept("(|CONCURRENTLY")) || + Matches("REPACK", "CONCURRENTLY", MatchAnyExcept("("))) COMPLETE_WITH("USING INDEX"); - /* If we have REPACK (*) <sth>, then add "USING INDEX" */ - else if (Matches("REPACK", "(*)", MatchAny)) + /* If we have REPACK (*) [ CONCURRENTLY ] <sth>, then add "USING INDEX" */ + else if (Matches("REPACK", "(*)", MatchAnyExcept("CONCURRENTLY")) || + Matches("REPACK", "(*)", "CONCURRENTLY", MatchAnyExcept("("))) COMPLETE_WITH("USING INDEX"); - /* If we have REPACK <sth> USING, then add the index as well */ - else if (Matches("REPACK", MatchAny, "USING", "INDEX")) + + /* + * Complete ... [ (*) ] [ CONCURRENTLY ] <sth> USING INDEX, with a list of + * indexes for <sth>. + */ + else if (TailMatches(MatchAnyExcept("(|CONCURRENTLY"), "USING", "INDEX")) { set_completion_reference(prev3_wd); COMPLETE_WITH_SCHEMA_QUERY(Query_for_index_of_table); diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h index a2bd5a897f8..b82dd17a966 100644 --- a/src/include/access/heapam.h +++ b/src/include/access/heapam.h @@ -323,14 +323,15 @@ extern void heap_multi_insert(Relation relation, struct TupleTableSlot **slots, BulkInsertState bistate); extern TM_Result heap_delete(Relation relation, ItemPointer tid, CommandId cid, Snapshot crosscheck, bool wait, - struct TM_FailureData *tmfd, bool changingPart); + struct TM_FailureData *tmfd, bool changingPart, + bool wal_logical); extern void heap_finish_speculative(Relation relation, ItemPointer tid); extern void heap_abort_speculative(Relation relation, ItemPointer tid); extern TM_Result heap_update(Relation relation, ItemPointer otid, HeapTuple newtup, CommandId cid, Snapshot crosscheck, bool wait, struct TM_FailureData *tmfd, LockTupleMode *lockmode, - TU_UpdateIndexes *update_indexes); + TU_UpdateIndexes *update_indexes, bool wal_logical); extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple, CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy, bool follow_updates, @@ -411,6 +412,10 @@ extern HTSV_Result HeapTupleSatisfiesVacuumHorizon(HeapTuple htup, Buffer buffer TransactionId *dead_after); extern void HeapTupleSetHintBits(HeapTupleHeader tuple, Buffer buffer, uint16 infomask, TransactionId xid); +extern bool HeapTupleMVCCInserted(HeapTuple htup, Snapshot snapshot, + Buffer buffer); +extern bool HeapTupleMVCCNotDeleted(HeapTuple htup, Snapshot snapshot, + Buffer buffer); extern bool HeapTupleHeaderIsOnlyLocked(HeapTupleHeader tuple); extern bool HeapTupleIsSurelyDead(HeapTuple htup, struct GlobalVisState *vistest); diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index 277df6b3cf0..8d4af07f840 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -104,6 +104,8 @@ #define XLH_DELETE_CONTAINS_OLD_KEY (1<<2) #define XLH_DELETE_IS_SUPER (1<<3) #define XLH_DELETE_IS_PARTITION_MOVE (1<<4) +/* See heap_delete() */ +#define XLH_DELETE_NO_LOGICAL (1<<5) /* convenience macro for checking whether any form of old tuple was logged */ #define XLH_DELETE_CONTAINS_OLD \ diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h index 1c9e802a6b1..289b64edfd9 100644 --- a/src/include/access/tableam.h +++ b/src/include/access/tableam.h @@ -22,6 +22,7 @@ #include "access/xact.h" #include "commands/vacuum.h" #include "executor/tuptable.h" +#include "replication/logical.h" #include "storage/read_stream.h" #include "utils/rel.h" #include "utils/snapshot.h" @@ -623,6 +624,8 @@ typedef struct TableAmRoutine Relation OldIndex, bool use_sort, TransactionId OldestXmin, + Snapshot snapshot, + LogicalDecodingContext *decoding_ctx, TransactionId *xid_cutoff, MultiXactId *multi_cutoff, double *num_tuples, @@ -1627,6 +1630,10 @@ table_relation_copy_data(Relation rel, const RelFileLocator *newrlocator) * not needed for the relation's AM * - *xid_cutoff - ditto * - *multi_cutoff - ditto + * - snapshot - if != NULL, ignore data changes done by transactions that this + * (MVCC) snapshot considers still in-progress or in the future. + * - decoding_ctx - logical decoding context, to capture concurrent data + * changes. * * Output parameters: * - *xid_cutoff - rel's new relfrozenxid value, may be invalid @@ -1639,6 +1646,8 @@ table_relation_copy_for_cluster(Relation OldTable, Relation NewTable, Relation OldIndex, bool use_sort, TransactionId OldestXmin, + Snapshot snapshot, + LogicalDecodingContext *decoding_ctx, TransactionId *xid_cutoff, MultiXactId *multi_cutoff, double *num_tuples, @@ -1647,6 +1656,7 @@ table_relation_copy_for_cluster(Relation OldTable, Relation NewTable, { OldTable->rd_tableam->relation_copy_for_cluster(OldTable, NewTable, OldIndex, use_sort, OldestXmin, + snapshot, decoding_ctx, xid_cutoff, multi_cutoff, num_tuples, tups_vacuumed, tups_recently_dead); diff --git a/src/include/commands/cluster.h b/src/include/commands/cluster.h index 7f4138c7b36..532ffa7208d 100644 --- a/src/include/commands/cluster.h +++ b/src/include/commands/cluster.h @@ -13,10 +13,15 @@ #ifndef CLUSTER_H #define CLUSTER_H +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" +#include "replication/logical.h" #include "storage/lock.h" +#include "storage/relfilelocator.h" #include "utils/relcache.h" +#include "utils/resowner.h" +#include "utils/tuplestore.h" /* flag bits for ClusterParams->options */ @@ -24,6 +29,7 @@ #define CLUOPT_RECHECK 0x02 /* recheck relation state */ #define CLUOPT_RECHECK_ISCLUSTERED 0x04 /* recheck relation state for * indisclustered */ +#define CLUOPT_CONCURRENT 0x08 /* allow concurrent data changes */ /* options for CLUSTER */ typedef struct ClusterParams @@ -32,14 +38,95 @@ typedef struct ClusterParams } ClusterParams; +/* + * The following definitions are used by REPACK CONCURRENTLY. + */ + +extern RelFileLocator repacked_rel_locator; +extern RelFileLocator repacked_rel_toast_locator; + +typedef enum +{ + CHANGE_INSERT, + CHANGE_UPDATE_OLD, + CHANGE_UPDATE_NEW, + CHANGE_DELETE, + CHANGE_TRUNCATE +} ConcurrentChangeKind; + +typedef struct ConcurrentChange +{ + /* See the enum above. */ + ConcurrentChangeKind kind; + + /* + * The actual tuple. + * + * The tuple data follows the ConcurrentChange structure. Before use make + * sure the tuple is correctly aligned (ConcurrentChange can be stored as + * bytea) and that tuple->t_data is fixed. + */ + HeapTupleData tup_data; +} ConcurrentChange; + +#define SizeOfConcurrentChange (offsetof(ConcurrentChange, tup_data) + \ + sizeof(HeapTupleData)) + +/* + * Logical decoding state. + * + * Here we store the data changes that we decode from WAL while the table + * contents is being copied to a new storage. Also the necessary metadata + * needed to apply these changes to the table is stored here. + */ +typedef struct RepackDecodingState +{ + /* The relation whose changes we're decoding. */ + Oid relid; + + /* + * Decoded changes are stored here. Although we try to avoid excessive + * batches, it can happen that the changes need to be stored to disk. The + * tuplestore does this transparently. + */ + Tuplestorestate *tstore; + + /* The current number of changes in tstore. */ + double nchanges; + + /* + * Descriptor to store the ConcurrentChange structure serialized (bytea). + * We can't store the tuple directly because tuplestore only supports + * minimum tuple and we may need to transfer OID system column from the + * output plugin. Also we need to transfer the change kind, so it's better + * to put everything in the structure than to use 2 tuplestores "in + * parallel". + */ + TupleDesc tupdesc_change; + + /* Tuple descriptor needed to update indexes. */ + TupleDesc tupdesc; + + /* Slot to retrieve data from tstore. */ + TupleTableSlot *tsslot; + + ResourceOwner resowner; +} RepackDecodingState; + + + extern void ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel); extern void cluster_rel(RepackCommand command, bool usingindex, - Relation OldHeap, Oid indexOid, ClusterParams *params); + Relation OldHeap, Oid indexOid, ClusterParams *params, + bool isTopLevel); extern void check_index_is_clusterable(Relation OldHeap, Oid indexOid, LOCKMODE lockmode); extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); +extern void repack_decode_concurrent_changes(LogicalDecodingContext *ctx, + XLogRecPtr end_of_wal); + extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, @@ -47,6 +134,7 @@ extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool swap_toast_by_content, bool check_constraints, bool is_internal, + bool reindex, TransactionId frozenXid, MultiXactId cutoffMulti, char newrelpersistence); diff --git a/src/include/commands/progress.h b/src/include/commands/progress.h index 5b6639c114c..93917ad5544 100644 --- a/src/include/commands/progress.h +++ b/src/include/commands/progress.h @@ -59,18 +59,20 @@ /* * Progress parameters for REPACK. * - * Note: Since REPACK shares some code with CLUSTER, these values are also - * used by CLUSTER. (CLUSTER is now deprecated, so it makes little sense to - * introduce a separate set of constants.) + * Note: Since REPACK shares some code with CLUSTER, (some of) these values + * are also used by CLUSTER. (CLUSTER is now deprecated, so it makes little + * sense to introduce a separate set of constants.) */ #define PROGRESS_REPACK_COMMAND 0 #define PROGRESS_REPACK_PHASE 1 #define PROGRESS_REPACK_INDEX_RELID 2 #define PROGRESS_REPACK_HEAP_TUPLES_SCANNED 3 -#define PROGRESS_REPACK_HEAP_TUPLES_WRITTEN 4 -#define PROGRESS_REPACK_TOTAL_HEAP_BLKS 5 -#define PROGRESS_REPACK_HEAP_BLKS_SCANNED 6 -#define PROGRESS_REPACK_INDEX_REBUILD_COUNT 7 +#define PROGRESS_REPACK_HEAP_TUPLES_INSERTED 4 +#define PROGRESS_REPACK_HEAP_TUPLES_UPDATED 5 +#define PROGRESS_REPACK_HEAP_TUPLES_DELETED 6 +#define PROGRESS_REPACK_TOTAL_HEAP_BLKS 7 +#define PROGRESS_REPACK_HEAP_BLKS_SCANNED 8 +#define PROGRESS_REPACK_INDEX_REBUILD_COUNT 9 /* * Phases of repack (as advertised via PROGRESS_REPACK_PHASE). @@ -79,9 +81,10 @@ #define PROGRESS_REPACK_PHASE_INDEX_SCAN_HEAP 2 #define PROGRESS_REPACK_PHASE_SORT_TUPLES 3 #define PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP 4 -#define PROGRESS_REPACK_PHASE_SWAP_REL_FILES 5 -#define PROGRESS_REPACK_PHASE_REBUILD_INDEX 6 -#define PROGRESS_REPACK_PHASE_FINAL_CLEANUP 7 +#define PROGRESS_REPACK_PHASE_CATCH_UP 5 +#define PROGRESS_REPACK_PHASE_SWAP_REL_FILES 6 +#define PROGRESS_REPACK_PHASE_REBUILD_INDEX 7 +#define PROGRESS_REPACK_PHASE_FINAL_CLEANUP 8 /* * Commands of PROGRESS_REPACK diff --git a/src/include/replication/snapbuild.h b/src/include/replication/snapbuild.h index 6d4d2d1814c..802fc4b0823 100644 --- a/src/include/replication/snapbuild.h +++ b/src/include/replication/snapbuild.h @@ -73,6 +73,7 @@ extern void FreeSnapshotBuilder(SnapBuild *builder); extern void SnapBuildSnapDecRefcount(Snapshot snap); extern Snapshot SnapBuildInitialSnapshot(SnapBuild *builder); +extern Snapshot SnapBuildInitialSnapshotForRepack(SnapBuild *builder); extern Snapshot SnapBuildMVCCFromHistoric(Snapshot snapshot, bool in_place); extern const char *SnapBuildExportSnapshot(SnapBuild *builder); extern void SnapBuildClearExportedSnapshot(void); diff --git a/src/include/storage/lockdefs.h b/src/include/storage/lockdefs.h index 7f3ba0352f6..2739327b0da 100644 --- a/src/include/storage/lockdefs.h +++ b/src/include/storage/lockdefs.h @@ -36,8 +36,8 @@ typedef int LOCKMODE; #define AccessShareLock 1 /* SELECT */ #define RowShareLock 2 /* SELECT FOR UPDATE/FOR SHARE */ #define RowExclusiveLock 3 /* INSERT, UPDATE, DELETE */ -#define ShareUpdateExclusiveLock 4 /* VACUUM (non-FULL), ANALYZE, CREATE - * INDEX CONCURRENTLY */ +#define ShareUpdateExclusiveLock 4 /* VACUUM (non-exclusive), ANALYZE, CREATE + * INDEX CONCURRENTLY, REPACK CONCURRENTLY */ #define ShareLock 5 /* CREATE INDEX (WITHOUT CONCURRENTLY) */ #define ShareRowExclusiveLock 6 /* like EXCLUSIVE MODE, but allows ROW * SHARE */ diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index 147b190210a..5eeabdc6c4f 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -61,6 +61,8 @@ extern Snapshot GetLatestSnapshot(void); extern void SnapshotSetCommandId(CommandId curcid); extern Snapshot CopySnapshot(Snapshot snapshot); +extern void FreeSnapshot(Snapshot snapshot); + extern Snapshot GetCatalogSnapshot(Oid relid); extern Snapshot GetNonHistoricCatalogSnapshot(Oid relid); extern void InvalidateCatalogSnapshot(void); diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index fc82cd67f6c..f16422175f8 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -11,10 +11,11 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +# REGRESS = injection_points hashagg reindex_conc vacuum REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress -ISOLATION = basic inplace syscache-update-pruned +ISOLATION = basic inplace syscache-update-pruned repack +ISOLATION_OPTS = --temp-config $(top_srcdir)/src/test/modules/injection_points/logical.conf TAP_TESTS = 1 diff --git a/src/test/modules/injection_points/expected/repack.out b/src/test/modules/injection_points/expected/repack.out new file mode 100644 index 00000000000..b575e9052ee --- /dev/null +++ b/src/test/modules/injection_points/expected/repack.out @@ -0,0 +1,113 @@ +Parsed test spec with 2 sessions + +starting permutation: wait_before_lock change_existing change_new change_subxact1 change_subxact2 check2 wakeup_before_lock check1 +injection_points_attach +----------------------- + +(1 row) + +step wait_before_lock: + REPACK (CONCURRENTLY) repack_test USING INDEX repack_test_pkey; + <waiting ...> +step change_existing: + UPDATE repack_test SET i=10 where i=1; + UPDATE repack_test SET j=20 where i=2; + UPDATE repack_test SET i=30 where i=3; + UPDATE repack_test SET i=40 where i=30; + DELETE FROM repack_test WHERE i=4; + +step change_new: + INSERT INTO repack_test(i, j) VALUES (5, 5), (6, 6), (7, 7), (8, 8); + UPDATE repack_test SET i=50 where i=5; + UPDATE repack_test SET j=60 where i=6; + DELETE FROM repack_test WHERE i=7; + +step change_subxact1: + BEGIN; + INSERT INTO repack_test(i, j) VALUES (100, 100); + SAVEPOINT s1; + UPDATE repack_test SET i=101 where i=100; + SAVEPOINT s2; + UPDATE repack_test SET i=102 where i=101; + COMMIT; + +step change_subxact2: + BEGIN; + SAVEPOINT s1; + INSERT INTO repack_test(i, j) VALUES (110, 110); + ROLLBACK TO SAVEPOINT s1; + INSERT INTO repack_test(i, j) VALUES (110, 111); + COMMIT; + +step check2: + INSERT INTO relfilenodes(node) + SELECT relfilenode FROM pg_class WHERE relname='repack_test'; + + SELECT i, j FROM repack_test ORDER BY i, j; + + INSERT INTO data_s2(i, j) + SELECT i, j FROM repack_test; + + i| j +---+--- + 2| 20 + 6| 60 + 8| 8 + 10| 1 + 40| 3 + 50| 5 +102|100 +110|111 +(8 rows) + +step wakeup_before_lock: + SELECT injection_points_wakeup('repack-concurrently-before-lock'); + +injection_points_wakeup +----------------------- + +(1 row) + +step wait_before_lock: <... completed> +step check1: + INSERT INTO relfilenodes(node) + SELECT relfilenode FROM pg_class WHERE relname='repack_test'; + + SELECT count(DISTINCT node) FROM relfilenodes; + + SELECT i, j FROM repack_test ORDER BY i, j; + + INSERT INTO data_s1(i, j) + SELECT i, j FROM repack_test; + + SELECT count(*) + FROM data_s1 d1 FULL JOIN data_s2 d2 USING (i, j) + WHERE d1.i ISNULL OR d2.i ISNULL; + +count +----- + 2 +(1 row) + + i| j +---+--- + 2| 20 + 6| 60 + 8| 8 + 10| 1 + 40| 3 + 50| 5 +102|100 +110|111 +(8 rows) + +count +----- + 0 +(1 row) + +injection_points_detach +----------------------- + +(1 row) + diff --git a/src/test/modules/injection_points/logical.conf b/src/test/modules/injection_points/logical.conf new file mode 100644 index 00000000000..c8f264bc6cb --- /dev/null +++ b/src/test/modules/injection_points/logical.conf @@ -0,0 +1 @@ +wal_level = logical \ No newline at end of file diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 20390d6b4bf..29561103bbf 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -47,9 +47,13 @@ tests += { 'specs': [ 'basic', 'inplace', + 'repack', 'syscache-update-pruned', ], 'runningcheck': false, # see syscache-update-pruned + # 'repack' requires wal_level = 'logical'. + 'regress_args': ['--temp-config', files('logical.conf')], + }, 'tap': { 'env': { diff --git a/src/test/modules/injection_points/specs/repack.spec b/src/test/modules/injection_points/specs/repack.spec new file mode 100644 index 00000000000..75850334986 --- /dev/null +++ b/src/test/modules/injection_points/specs/repack.spec @@ -0,0 +1,143 @@ +# Prefix the system columns with underscore as they are not allowed as column +# names. +setup +{ + CREATE EXTENSION injection_points; + + CREATE TABLE repack_test(i int PRIMARY KEY, j int); + INSERT INTO repack_test(i, j) VALUES (1, 1), (2, 2), (3, 3), (4, 4); + + CREATE TABLE relfilenodes(node oid); + + CREATE TABLE data_s1(i int, j int); + CREATE TABLE data_s2(i int, j int); +} + +teardown +{ + DROP TABLE repack_test; + DROP EXTENSION injection_points; + + DROP TABLE relfilenodes; + DROP TABLE data_s1; + DROP TABLE data_s2; +} + +session s1 +setup +{ + SELECT injection_points_set_local(); + SELECT injection_points_attach('repack-concurrently-before-lock', 'wait'); +} +# Perform the initial load and wait for s2 to do some data changes. +step wait_before_lock +{ + REPACK (CONCURRENTLY) repack_test USING INDEX repack_test_pkey; +} +# Check the table from the perspective of s1. +# +# Besides the contents, we also check that relfilenode has changed. + +# Have each session write the contents into a table and use FULL JOIN to check +# if the outputs are identical. +step check1 +{ + INSERT INTO relfilenodes(node) + SELECT relfilenode FROM pg_class WHERE relname='repack_test'; + + SELECT count(DISTINCT node) FROM relfilenodes; + + SELECT i, j FROM repack_test ORDER BY i, j; + + INSERT INTO data_s1(i, j) + SELECT i, j FROM repack_test; + + SELECT count(*) + FROM data_s1 d1 FULL JOIN data_s2 d2 USING (i, j) + WHERE d1.i ISNULL OR d2.i ISNULL; +} +teardown +{ + SELECT injection_points_detach('repack-concurrently-before-lock'); +} + +session s2 +# Change the existing data. UPDATE changes both key and non-key columns. Also +# update one row twice to test whether tuple version generated by this session +# can be found. +step change_existing +{ + UPDATE repack_test SET i=10 where i=1; + UPDATE repack_test SET j=20 where i=2; + UPDATE repack_test SET i=30 where i=3; + UPDATE repack_test SET i=40 where i=30; + DELETE FROM repack_test WHERE i=4; +} +# Insert new rows and UPDATE / DELETE some of them. Again, update both key and +# non-key column. +step change_new +{ + INSERT INTO repack_test(i, j) VALUES (5, 5), (6, 6), (7, 7), (8, 8); + UPDATE repack_test SET i=50 where i=5; + UPDATE repack_test SET j=60 where i=6; + DELETE FROM repack_test WHERE i=7; +} + +# When applying concurrent data changes, we should see the effects of an +# in-progress subtransaction. +# +# XXX Not sure this test is useful now - it was designed for the patch that +# preserves tuple visibility and which therefore modifies +# TransactionIdIsCurrentTransactionId(). +step change_subxact1 +{ + BEGIN; + INSERT INTO repack_test(i, j) VALUES (100, 100); + SAVEPOINT s1; + UPDATE repack_test SET i=101 where i=100; + SAVEPOINT s2; + UPDATE repack_test SET i=102 where i=101; + COMMIT; +} + +# When applying concurrent data changes, we should not see the effects of a +# rolled back subtransaction. +# +# XXX Is this test useful? See above. +step change_subxact2 +{ + BEGIN; + SAVEPOINT s1; + INSERT INTO repack_test(i, j) VALUES (110, 110); + ROLLBACK TO SAVEPOINT s1; + INSERT INTO repack_test(i, j) VALUES (110, 111); + COMMIT; +} + +# Check the table from the perspective of s2. +step check2 +{ + INSERT INTO relfilenodes(node) + SELECT relfilenode FROM pg_class WHERE relname='repack_test'; + + SELECT i, j FROM repack_test ORDER BY i, j; + + INSERT INTO data_s2(i, j) + SELECT i, j FROM repack_test; +} +step wakeup_before_lock +{ + SELECT injection_points_wakeup('repack-concurrently-before-lock'); +} + +# Test if data changes introduced while one session is performing REPACK +# CONCURRENTLY find their way into the table. +permutation + wait_before_lock + change_existing + change_new + change_subxact1 + change_subxact2 + check2 + wakeup_before_lock + check1 diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index 3a1d1d28282..fe227bd8a30 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1999,17 +1999,17 @@ pg_stat_progress_cluster| SELECT s.pid, WHEN 2 THEN 'index scanning heap'::text WHEN 3 THEN 'sorting tuples'::text WHEN 4 THEN 'writing new heap'::text - WHEN 5 THEN 'swapping relation files'::text - WHEN 6 THEN 'rebuilding index'::text - WHEN 7 THEN 'performing final cleanup'::text + WHEN 6 THEN 'swapping relation files'::text + WHEN 7 THEN 'rebuilding index'::text + WHEN 8 THEN 'performing final cleanup'::text ELSE NULL::text END AS phase, (s.param3)::oid AS cluster_index_relid, s.param4 AS heap_tuples_scanned, s.param5 AS heap_tuples_written, - s.param6 AS heap_blks_total, - s.param7 AS heap_blks_scanned, - s.param8 AS index_rebuild_count + s.param8 AS heap_blks_total, + s.param9 AS heap_blks_scanned, + s.param10 AS index_rebuild_count FROM (pg_stat_get_progress_info('CLUSTER'::text) s(pid, datid, relid, param1, param2, param3, param4, param5, param6, param7, param8, param9, param10, param11, param12, param13, param14, param15, param16, param17, param18, param19, param20) LEFT JOIN pg_database d ON ((s.datid = d.oid))); pg_stat_progress_copy| SELECT s.pid, @@ -2081,17 +2081,20 @@ pg_stat_progress_repack| SELECT s.pid, WHEN 2 THEN 'index scanning heap'::text WHEN 3 THEN 'sorting tuples'::text WHEN 4 THEN 'writing new heap'::text - WHEN 5 THEN 'swapping relation files'::text - WHEN 6 THEN 'rebuilding index'::text - WHEN 7 THEN 'performing final cleanup'::text + WHEN 5 THEN 'catch-up'::text + WHEN 6 THEN 'swapping relation files'::text + WHEN 7 THEN 'rebuilding index'::text + WHEN 8 THEN 'performing final cleanup'::text ELSE NULL::text END AS phase, (s.param3)::oid AS repack_index_relid, s.param4 AS heap_tuples_scanned, - s.param5 AS heap_tuples_written, - s.param6 AS heap_blks_total, - s.param7 AS heap_blks_scanned, - s.param8 AS index_rebuild_count + s.param5 AS heap_tuples_inserted, + s.param6 AS heap_tuples_updated, + s.param7 AS heap_tuples_deleted, + s.param8 AS heap_blks_total, + s.param9 AS heap_blks_scanned, + s.param10 AS index_rebuild_count FROM (pg_stat_get_progress_info('REPACK'::text) s(pid, datid, relid, param1, param2, param3, param4, param5, param6, param7, param8, param9, param10, param11, param12, param13, param14, param15, param16, param17, param18, param19, param20) LEFT JOIN pg_database d ON ((s.datid = d.oid))); pg_stat_progress_vacuum| SELECT s.pid, diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 24f98ed1e9e..af967625181 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -486,6 +486,8 @@ CompressFileHandle CompressionLocation CompressorState ComputeXidHorizonsResult +ConcurrentChange +ConcurrentChangeKind ConditionVariable ConditionVariableMinimallyPadded ConditionalStack @@ -1258,6 +1260,7 @@ IndexElem IndexFetchHeapData IndexFetchTableData IndexInfo +IndexInsertState IndexList IndexOnlyScan IndexOnlyScanState @@ -2538,6 +2541,7 @@ ReorderBufferUpdateProgressTxnCB ReorderTuple RepOriginId RepackCommand +RepackDecodingState RepackStmt ReparameterizeForeignPathByChild_function ReplaceVarsFromTargetList_context -- 2.47.1 --=-=-=-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v1 1/2] Support changing a column into a stored generated column @ 2026-03-16 23:25 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-16 23:25 UTC (permalink / raw) This adds basic support for an ALTER TABLE ... ALTER COLUMN command to turn a regular column into a stored generated column. The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. Since this is a first prototype, no thought has been given to partitioned nor foreign tables, so these are not supported either. This operation always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 137 +++++++++++++++++- src/backend/parser/gram.y | 31 ++++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 2 + src/test/regress/expected/alter_table.out | 122 ++++++++++++++++ src/test/regress/sql/alter_table.sql | 69 +++++++++ 6 files changed, 361 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 67e42e5df29..e7386e81b07 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,10 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4746,6 +4750,7 @@ AlterTableGetLockLevel(List *cmds) case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: + case AT_AddGeneratedAsExprStored: cmd_lockmode = AccessExclusiveLock; break; @@ -5321,6 +5326,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* No command-specific prep needed */ pass = AT_PASS_MISC; break; + case AT_AddGeneratedAsExprStored: + /* No support yet for: partitioned tables, foreign tables */ + ATSimplePermissions(cmd->subtype, rel, ATT_TABLE); + + /* + * This has similar mechanics to AT_SetExpression, let's use the + * same pass. + */ + pass = AT_PASS_SET_EXPRESSION; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -5733,6 +5748,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, ATExecSplitPartition(wqueue, tab, rel, (PartitionCmd *) cmd->def, context); break; + case AT_AddGeneratedAsExprStored: + cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode, + cur_pass, context); + Assert(cmd != NULL); + address = ATExecAddGeneratedAsExprStored(tab, rel, cmd->name, (Constraint *) cmd->def); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -6785,6 +6806,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... DROP IDENTITY"; case AT_ReAddStatistics: return NULL; /* not real grammar */ + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; } return NULL; @@ -8823,6 +8846,118 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15425,7 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index c2584249603..74440e801d4 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index ffadd667167..6b61513e6d0 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2568,6 +2568,7 @@ typedef enum AlterTableType AT_SetIdentity, /* SET identity column options */ AT_DropIdentity, /* DROP IDENTITY */ AT_ReAddStatistics, /* internal to commands/tablecmds.c */ + AT_AddGeneratedAsExprStored, /* ADD GENERATED ALWAYS AS (...) STORED */ } AlterTableType; typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..7c1699d538c 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -315,6 +315,8 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_ReAddStatistics: strtype = "(re) ADD STATS"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; } if (subcmd->recurse) diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..75f64628aef 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,125 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; +ERROR: ALTER action ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED cannot be performed on relation "tpart" +DETAIL: This operation is not supported for partitioned tables. +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 +drop cascades to table testgen.t2 +drop cascades to table testgen.tpart diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..d776595a6ed 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,72 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; + +-- not supported: partitioned tables +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart alter column b add generated always as (a * 2) stored; + +drop schema testgen cascade; -- 2.51.2 --24dzbv6kpqxe4yje Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v1-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v2 1/2] Support changing a column into a stored generated column @ 2026-03-29 19:45 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-03-29 19:45 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 187 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 423 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c69c12dc014..66622bf4837 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -760,6 +760,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4743,6 +4751,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5067,6 +5076,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5461,6 +5477,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6667,6 +6689,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8823,6 +8847,165 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, + bool recursing, LOCKMODE lockmode +) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15290,7 +15473,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0fea726cdd5..315ee7d94e2 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2717,6 +2717,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index df431220ac5..74958ef0dfa 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2506,6 +2506,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ccd79dfecc0..2567d918ec3 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4863,3 +4863,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 01d58d7e3ff3f7482ff478cb4a49c48aad276138 -- 2.47.0 --tyerjrpxgsfvwown Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v2-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v3 1/2] Support changing a column into a stored generated column @ 2026-04-24 08:44 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-04-24 08:44 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 186 +++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 126 ++++++++++++ src/test/regress/sql/alter_table.sql | 76 +++++++ 6 files changed, 422 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d8d7969bf30..aa54c629f8b 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,164 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + Expr *defval; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + AddRelationNewConstraints(rel, list_make1(rawEnt), NIL, + false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Build a concrete expression for the new default (generated) value */ + defval = (Expr *) build_column_default(rel, attnum); + defval = expression_planner(defval); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = defval; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15502,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..08981f4e380 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,129 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table testgen.t3 +drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..76187083289 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,79 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 3b28dad70e2fa57a973697d51242c284d475c7df -- 2.47.0 --4wx636ozsu2eakgd Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename="v3-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v4 1/2] Support changing a column into a stored generated column @ 2026-05-14 21:51 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-05-14 21:51 UTC (permalink / raw) This adds an ALTER TABLE subcommand to turn a regular column into a stored generated column: ... ALTER COLUMN c ADD GENERATED ALWAYS as (expr) STORED The syntax is chosen to be similar to ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION. Phase 2 happens in the same pass as the former, in order to run the cleanup code in ATPostAlterTypeCleanup, without which for example we would not re-check constraints when rewriting the table. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. There is one limitation: currently DROP EXPRESSION does not allow to change an inheritance tree of depth > 2. This seems like an oversight, but in order to not feature-creep this commit, this is postponed for later; it should then be fixed for both DROP EXPRESSION and this new command. This is mostly useful as a first step to be able to add a stored generated column without rewriting the table under an exclusive lock. For ease of review, the operation as of this commit always rewrites the contents of the column using the new generated expression. --- src/backend/commands/tablecmds.c | 194 ++++++++++++++++- src/backend/parser/gram.y | 31 +++ src/include/nodes/parsenodes.h | 1 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 200 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 142 +++++++++++++ 6 files changed, 570 insertions(+), 1 deletion(-) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 92b0f38c353..39faef0a114 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -783,6 +783,14 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static ObjectAddress ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4769,6 +4777,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedAsExprStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5093,6 +5102,13 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedAsExprStored: + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGeneratedAsExprStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_SET_EXPRESSION; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5487,6 +5503,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedAsExprStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedAsExprStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6695,6 +6717,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedAsExprStored: + return "ALTER COLUMN ... ADD GENERATED ALWAYS AS (...) STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8851,6 +8875,172 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation for + * + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + * + * Checks whether recursion is allowed, following the same logic as ATPrepDropExpression. + */ +static void +ATPrepAddGeneratedAsExprStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. See ATPrepDropExpression. + */ + if (!recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * ALTER TABLE ALTER COLUMN ADD GENERATED ALWAYS AS expr STORED + */ +static ObjectAddress +ATExecAddGeneratedAsExprStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + ObjectAddress address; + NewColumnValue *newval; + RawColumnDefault *rawEnt; + Relation pg_attribute; + List *newcons; + CookedConstraint *cookedDef; + + Assert(def->raw_expr != NULL); + Assert(def->cooked_expr == NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects. + */ + RememberAllDependentForRebuilding(tab, AT_AddGeneratedAsExprStored, + rel, attnum, colName); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + /* Make above changes visible */ + CommandCounterIncrement(); + + ReleaseSysCache(tuple); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawEnt = palloc_object(RawColumnDefault); + rawEnt->attnum = attnum; + rawEnt->raw_default = def->raw_expr; + rawEnt->generated = def->generated_kind; + newcons = AddRelationNewConstraints(rel, list_make1(rawEnt), + NIL, false, true, false, NULL); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* + * At the moment, AddRelationNewConstraints always returns one element + * when called with a generated = STORED input, but guard against + * accessing an empty list anyway. + */ + if (list_length(newcons) < 1) + ereport(ERROR, + errmsg_internal("expected exactly one processed default value")); + + cookedDef = linitial(newcons); + + /* + * Clear all the missing values if we're rewriting the table, since this + * renders them pointless. + */ + RelationClearMissing(rel); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Drop any pg_statistic entry for the column */ + RemoveStatistics(RelationGetRelid(rel), attnum); + + /* Schedule a rewrite */ + newval = palloc0_object(NewColumnValue); + newval->attnum = attnum; + newval->expr = (Expr *) cookedDef->expr; + newval->is_generated = true; + tab->newvals = lappend(tab->newvals, newval); + tab->rewrite |= AT_REWRITE_DEFAULT_VAL; + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ @@ -15320,7 +15510,9 @@ RememberAllDependentForRebuilding(AlteredTableInfo *tab, AlterTableType subtype, SysScanDesc scan; HeapTuple depTup; - Assert(subtype == AT_AlterColumnType || subtype == AT_SetExpression); + Assert(subtype == AT_AlterColumnType + || subtype == AT_SetExpression + || subtype == AT_AddGeneratedAsExprStored); depRel = table_open(DependRelationId, RowExclusiveLock); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..8cd75572257 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,37 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS AS ( <expression> ) STORED */ + | ALTER opt_column ColId ADD_P GENERATED generated_when AS '(' a_expr ')' STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->raw_expr = $9; + c->cooked_expr = NULL; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @5; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddGeneratedAsExprStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 91377a6cde3..620e2dd73bf 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2527,6 +2527,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedAsExprStored, /* add generated always as (...) stored */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..3132ceac61f 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedAsExprStored: + strtype = "ADD GENERATED ALWAYS AS (...) STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 6dd22be0e8d..2d0ce414d12 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4875,3 +4875,203 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +ERROR: check constraint "chk_gen_clause" of relation "t2" is violated by some row +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; + did_not_rewrite +----------------- + t +(1 row) + +\d+ testgen.t2 + Table "testgen.t2" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t2_b_not_null" NOT NULL "b" + +drop table testgen.t2; +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; + did_rewrite_idx +----------------- + t +(1 row) + +drop table testgen.t3; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +ERROR: ALTER TABLE / ADD GENERATED ALWAYS AS (expr) STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; +ERROR: column "doesnotexist" does not exist +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default as (a * 2) stored; + ^ +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +ERROR: syntax error at or near ";" +LINE 1: ...e testgen.t1 alter column b add generated always as (a * 2); + ^ +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...n.t1 alter column b add generated always as (a * 2) virtual; + ^ +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +ERROR: generation expression is not immutable +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +ERROR: cannot use subquery in column generation expression +drop table testgen.t3; +drop schema testgen cascade; +NOTICE: drop cascades to table testgen.t1 diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..51d818d4995 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,145 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS as ( expr ) STORED +-- turning a regular column into a stored generated column +create schema testgen; + +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) + select x, x from generate_series(1, 10) x; +alter table testgen.t1 alter column b + add generated always as (a * 2) stored; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when another constraint conflicts with the new expression +create table testgen.t2 (a int, b int not null); +insert into testgen.t2 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t2 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t2') as t2_filenode_before \gset +alter table testgen.t2 alter column b add generated always as (a * 3) stored; +select pg_relation_filenode('testgen.t2') as t2_filenode_after \gset +select :t2_filenode_before = :t2_filenode_after as did_not_rewrite; +\d+ testgen.t2 +drop table testgen.t2; + +-- rewrite an indexed column +create table testgen.t3 (a int, b int); +create index idx_b on testgen.t3 (b); +insert into testgen.t3 (a, b) select x, x from generate_series(1, 10) x; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_before \gset +alter table testgen.t3 alter column b add generated always as (a * 2) stored; +select pg_relation_filenode('testgen.idx_b') as idx_filenode_after \gset +select :idx_filenode_before != :idx_filenode_after as did_rewrite_idx; +drop table testgen.t3; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always as (a * 2) stored; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always as (a * 2) stored; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always as (a * 2) stored; +rollback; + +drop table testgen.tpart; + +-- subpartitions +create table testgen.tpart (a int, b int, c int) + partition by hash (a); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0) + partition by hash (b); +create table testgen.tpart_p1_1 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p1_2 partition of testgen.tpart_p1 + for values with (modulus 2, remainder 1); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1) + partition by hash (b); +create table testgen.tpart_p2_1 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2_2 partition of testgen.tpart_p2 + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) +select x, y +from generate_series(1, 5) x + cross join generate_series(1, 5) y; +-- currently, it is not possible to change the generated state of an +-- inheritance tree of depth >= 2 (same as in DROP EXPRESSION), so we expect an +-- error here. This might be fixed later. +begin; +alter table testgen.tpart alter column c + add generated always as (a + b) stored; +rollback; + +drop table testgen.tpart; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always as (bar * 2) stored; + +create table testgen.t1 (a int); + +alter table testgen.t1 alter column doesnotexist + add generated always as (bar * 2) stored; + +alter table testgen.t1 add column b int; + +alter table testgen.t1 alter column b + add generated always as (doesnotexist * 2) stored; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default as (a * 2) stored; + +-- invalid: only supports STORED +alter table testgen.t1 alter column b add generated always as (a * 2); +alter table testgen.t1 alter column b add generated always as (a * 2) virtual; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always as (a * 2) stored; +drop table testgen.t2; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 alter column b + add generated always as (a + random()) stored; +-- invalid: expr cannot use subselects +alter table testgen.t3 alter column b + add generated always as (a + (select 1)) stored; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6b48f5d1a74168c78badfb2e59ef788bb8eb396e -- 2.47.0 --zfrmj4necy5zy3mo Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="v4-0002-Try-to-avoid-a-rewrite-when-adding-a-stored-gener.patch" ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v5] Support changing a column into a stored generated column @ 2026-06-30 13:21 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-06-30 13:21 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 468 +++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 490 ++++++++++++++++++ src/test/regress/sql/alter_table.sql | 314 +++++++++++ 9 files changed, 1404 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6dd518752c0..362096bfa9b 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 472db112fa7..a5a809f49d0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -790,6 +790,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4776,6 +4791,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5100,6 +5116,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5494,6 +5518,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6702,6 +6732,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8833,6 +8865,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 46b9add0604..f318287edd2 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1898,6 +1907,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2984,12 +2994,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 4133c404a6b..91c5630b98f 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2524,6 +2524,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index b891d68d4a7..caee39f773a 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,493 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..4a79e3b7219 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,317 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- check that the table isn't being scanned during phase 3, even if other +-- objects depend on the column we are changing. Lowering client_min_messages +-- makes the message 'verifying table...' be shown here when that happens. +create table testgen.t6 (a int, b int not null); +insert into testgen.t6 (a, b) values (1, 2); +alter table testgen.t6 add constraint c1 check (b > 0); +alter table testgen.t6 add constraint c2 check (b = a * 2); +create index on testgen.t6 (b); +set client_min_messages = 'DEBUG1'; +alter table testgen.t6 alter b + add generated always stored using constraint c2; +-- we expect to *not* see a "verifying table" message here +reset client_min_messages; +drop table testgen.t6; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: c776550e4662385b0ebeac653ae86755008d29f3 -- 2.47.0 --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="039_stored_generated_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tests the configuration where the public= ation is set up to publish=0A# stored generated columns.=0A#=0A# When publi= shing stored generated columns, it is not supported for the same=0A# column= to be generated on both the publisher and the subscriber. The only=0A# val= id configuration is for the column to be a regular column on the side of=0A= # the subscriber.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause = PostgreSQL::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;= =0A=0Amy $node_publisher =3D PostgreSQL::Test::Cluster->new('publisher');= =0A$node_publisher->init(allows_streaming =3D> 'logical');=0A$node_publishe= r->start;=0A=0Amy $node_subscriber =3D PostgreSQL::Test::Cluster->new('subs= criber');=0A$node_subscriber->init;=0A$node_subscriber->start;=0A=0A# We wi= ll use the same user throughout the test, so let's just fix it here.=0Asub = sql=0A{=0A local $Carp::CarpLevel =3D $Carp::CarpLevel + 1;=0A my ($node, $= sql_code) =3D @_;=0A $node->safe_psql('postgres', $sql_code);=0A}=0A=0A# Sc= hema and replication setup=0Amy $schema_ddl =3D qq[=0A CREATE SCHEMA sch1;= =0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($node_publisher,= $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Amy $publisher_con= nstr =3D $node_publisher->connstr . ' dbname=3Dpostgres';=0Asql($node_publi= sher, qq[=0A CREATE PUBLICATION tap_pub_schema FOR TABLES IN SCHEMA sch1=0A= WITH (publish_generated_columns =3D stored);=0A]);=0A=0Asql($node_subscrib= er, qq[=0A CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connst= r'=0A PUBLICATION tap_pub_schema;=0A]);=0A=0A$node_subscriber->wait_for_su= bscription_sync($node_publisher, 'tap_sub_schema');=0A=0A# Initial data=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (1)]);=0A$node_publ= isher->wait_for_catchup('tap_sub_schema');=0A=0A# Non-locking migration to = add a stored generated column=0A# b =3D (a * 2)=0Asql($node_subscriber, qq[= =0A ALTER TABLE sch1.tab1 ADD COLUMN b INT;=0A]);=0Asql($node_publisher, qq= [=0A INSERT INTO sch1.tab1 (a) VALUES (2);=0A ALTER TABLE sch1.tab1 ADD COL= UMN b INT;=0A INSERT INTO sch1.tab1 (a) VALUES (3);=0A]);=0A=0Asql($node_pu= blisher, qq[=0A -- Take care of new and updated rows, first.=0A CREATE FUNC= TION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN= =0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A CREATE TRIGGER tr= ig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTI= ON sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CONSTRAINT check_gen = CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A=0A INSERT INTO sch1.tab1= (a) VALUES (4);=0A=0A -- Now, backfill the table. In production, this migh= t be done in batches.=0A UPDATE sch1.tab1 SET b =3D a * 2 WHERE b IS NULL;= =0A=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A=0A INSERT IN= TO sch1.tab1 (a) VALUES (5);=0A=0A -- Now, we can convert the column withou= t a rewrite while holding an AccessExclusiveLock.=0A ALTER TABLE sch1.tab1 = ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A= =0A INSERT INTO sch1.tab1 (a) VALUES (6);=0A]);=0A=0A$node_publisher->wait_= for_catchup('tap_sub_schema');=0A=0Asql($node_subscriber, qq[=0A ALTER TABL= E sch1.tab1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) N= OT VALID;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0As= ql($node_publisher, qq[INSERT INTO sch1.tab1 (a) VALUES (7)]);=0A=0A$node_p= ublisher->wait_for_catchup('tap_sub_schema');=0A=0Amy $result =3D sql($node= _subscriber, qq[SELECT a, b FROM sch1.tab1 ORDER BY a]);=0Ais($result, qq[1= |2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12=0A7|14], 'check: fully replicated');=0A= =0A$node_subscriber->stop('fast');=0A$node_publisher->stop('fast');=0A=0Ado= ne_testing();=0A --vwkr5fjnqh6lckov Content-Type: application/x-perl Content-Disposition: attachment; filename="040_stored_generated_not_published.pl" Content-Transfer-Encoding: quoted-printable =0A# Copyright (c) 2021-2026, PostgreSQL Global Development Group=0A=0A# Lo= gical replication tests for=0A# ALTER COLUMN c ADD GENERATED ALWAYS STORED = USING CONSTRAINT name=0A#=0A# This tries to exercise the replication code p= aths when the column is=0A# converted to be a stored generated column and t= he publication is set up to=0A# *not* publish them. The scenario is what a = DBA would do to add a stored=0A# generated column without taking the table = offline.=0A#=0Ause strict;=0Ause warnings FATAL =3D> 'all';=0Ause PostgreSQ= L::Test::Cluster;=0Ause PostgreSQL::Test::Utils;=0Ause Test::More;=0A=0A# I= nitialize publisher node=0Amy $node_publisher =3D PostgreSQL::Test::Cluster= ->new('publisher');=0A$node_publisher->init(allows_streaming =3D> 'logical'= );=0A$node_publisher->start;=0A=0A# Create subscriber node=0Amy $node_subsc= riber =3D PostgreSQL::Test::Cluster->new('subscriber');=0A$node_subscriber-= >init;=0A$node_subscriber->start;=0A=0A# We will use the same user througho= ut the test, so let's just fix it here.=0Asub sql=0A{=0A local $Carp::CarpL= evel =3D $Carp::CarpLevel + 1;=0A my ($node, $sql_code) =3D @_;=0A $node->s= afe_psql('postgres', $sql_code);=0A}=0A=0Amy $schema_ddl =3D qq[=0A CREATE = SCHEMA sch1;=0A CREATE TABLE sch1.tab1 (a INT PRIMARY KEY);=0A];=0Asql($nod= e_publisher, $schema_ddl);=0Asql($node_subscriber, $schema_ddl);=0A=0Asql($= node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (1);=0A]);=0A=0A# S= et up replication=0Amy $publisher_connstr =3D $node_publisher->connstr . ' = dbname=3Dpostgres';=0Asql($node_publisher, qq[=0A CREATE PUBLICATION tap_pu= b_schema FOR TABLES IN SCHEMA sch1=0A WITH (publish_generated_columns =3D n= one)=0A]);=0Asql($node_subscriber, qq[=0A CREATE SUBSCRIPTION tap_sub_schem= a CONNECTION '$publisher_connstr'=0A PUBLICATION tap_pub_schema=0A]);= =0A$node_subscriber->wait_for_subscription_sync($node_publisher,=0A 'tap_su= b_schema');=0A=0Amy $result =3D sql($node_subscriber, "SELECT a FROM sch1.t= ab1");=0Ais($result, "1",=0A 'sanity check: initial data has been synced');= =0A=0A# Now we want to add a stored generated column `b`. Since we are not = set up=0A# to publish stored generated column, we need to set up the subscr= iber first.=0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ADD COLUMN= b INT;=0A]);=0Asql($node_publisher, qq[=0A ALTER TABLE sch1.tab1 ADD COLUM= N b INT;=0A]);=0A=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) v= alues (2);=0A]);=0A=0A# Take care of new/updated rows=0Asql($node_publisher= , qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS TRIGGER LANGUAGE plpgsq= l AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN NEW;=0A END=0A \$\$;=0A= CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON sch1.tab1=0A FOR EACH = ROW EXECUTE FUNCTION sch1.generate_b();=0A=0A ALTER TABLE sch1.tab1 ADD CON= STRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;=0A]);=0A= =0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (3);=0A]);= =0A=0A# Backfill=0Asql($node_publisher, qq[=0A UPDATE sch1.tab1 SET b =3D a= * 2 WHERE b IS NULL;=0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_ge= n;=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');=0A=0A# When= we switch "b" to a stored gen column on the publisher, it will not be=0A# = synced anymore. Let's set up the replica first, in order to not lose data.= =0Asql($node_subscriber, qq[=0A CREATE FUNCTION sch1.generate_b () RETURNS = TRIGGER LANGUAGE plpgsql AS \$\$=0A BEGIN=0A NEW.b =3D NEW.a * 2; RETURN = NEW;=0A END=0A \$\$;=0A CREATE TRIGGER trig_gen BEFORE INSERT OR UPDATE ON = sch1.tab1=0A FOR EACH ROW EXECUTE FUNCTION sch1.generate_b();=0A ALTER = TABLE sch1.tab1 ENABLE REPLICA TRIGGER trig_gen;=0A=0A ALTER TABLE sch1.tab= 1 ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM a * 2) NOT VALID;= =0A ALTER TABLE sch1.tab1 VALIDATE CONSTRAINT check_gen;=0A]);=0A=0Asql($no= de_publisher, qq[=0A INSERT INTO sch1.tab1 (a) values (4);=0A]);=0A=0A# Con= vert b to a stored generated column on the publisher first: b will not=0A# = be synced anymore, but the trigger and constraint on the subscriber guarant= ee=0A# that it writes the correct values.=0Asql($node_publisher, qq[=0A ALT= ER TABLE sch1.tab1 ALTER COLUMN b ADD GENERATED ALWAYS STORED USING CONSTRA= INT check_gen;=0A]);=0Asql($node_publisher, qq[=0A INSERT INTO sch1.tab1 (a= ) values (5);=0A]);=0A$node_publisher->wait_for_catchup('tap_sub_schema');= =0Asql($node_subscriber, qq[=0A ALTER TABLE sch1.tab1 ALTER COLUMN b ADD GE= NERATED ALWAYS STORED USING CONSTRAINT check_gen;=0A]);=0A=0Asql($node_publ= isher, qq[=0A INSERT INTO sch1.tab1 (a) values (6);=0A]);=0A=0A$node_publis= her->wait_for_catchup('tap_sub_schema');=0A=0A# Check: all the rows have th= e correct values in b=0A$result =3D sql($node_subscriber, "SELECT * FROM sc= h1.tab1 ORDER BY a");=0Ais($result, qq[1|2=0A2|4=0A3|6=0A4|8=0A5|10=0A6|12]= , 'check: all of "b" has been replicated');=0A=0A$node_subscriber->stop('fa= st');=0A$node_publisher->stop('fast');=0A=0Adone_testing();=0A --vwkr5fjnqh6lckov-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
* [PATCH v6] Support changing a column into a stored generated column @ 2026-07-03 05:52 Alberto Piai <[email protected]> 0 siblings, 0 replies; 276+ messages in thread From: Alberto Piai @ 2026-07-03 05:52 UTC (permalink / raw) This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED ALWAYS STORED USING CONSTRAINT constr_name The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data according to the intended generation expression, and adding a CHECK constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation. If the column is nullable, the constraint must be of the form: CHECK (column_name IS NOT DISTINCT FROM expr) if the column is NOT NULL, either of the following is acceptable: CHECK (column_name IS NOT DISTINCT FROM expr) CHECK (column_name = expr) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that in this case, since we're dealing with a generated column, only ALWAYS is supported. Additionally, STORED must always be specified. This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. Partitioning/inheritance is supported in the same way as DROP EXPRESSION: it is allowed to change the whole inheritace tree to/from a generated column at once; it is forbidden to change the parent table ONLY and it is forbidden to change a partition directly. See 8bf6ec3ba3a44448817af47a080587f3b71bee08 and the associated discussion. --- doc/src/sgml/ref/alter_table.sgml | 43 ++ src/backend/commands/tablecmds.c | 479 ++++++++++++++++++ src/backend/parser/gram.y | 30 ++ src/bin/psql/t/010_tab_completion.pl | 19 + src/bin/psql/tab-complete.in.c | 38 +- src/include/nodes/parsenodes.h | 2 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 476 +++++++++++++++++ src/test/regress/sql/alter_table.sql | 299 +++++++++++ 13 files changed, 1448 insertions(+), 4 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index ff7071bef5b..b95a43e1699 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -52,6 +52,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED ALWAYS STORED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -272,6 +273,48 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED ALWAYS STORED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given constraint. The constraint must be a + <literal>CHECK</literal> constraint proving that the values of the + column already satisfy the generation expression. The operation will + then be performed without rewriting the table. + </para> + + <para> + The main purpose of this form is to allow adding a stored generated + column to a table without performing a table rewrite while holding an + <literal>ACCESS EXCLUSIVE</literal> lock. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + If the column being modified is nullable, the constraint must be of the + form <literal>CHECK (column_name IS NOT DISTINCT FROM expr)</literal>. + If the column is <literal>NOT NULL</literal>, then the form + <literal>CHECK (column_name = expr)</literal> is also allowed. + </para> + + <para> + After this command is run, <literal>column_name</literal> will be a stored + generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 95abaf4890c..1b04660c8c6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -101,6 +101,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -790,6 +791,21 @@ static void ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, AlterTableUtilityContext *context); +static void ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode); +static void checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); static List *collectPartitionIndexExtDeps(List *partitionOids); static void applyPartitionIndexExtDeps(Oid newPartOid, List *extDepState); static void freePartitionIndexExtDeps(List *extDepState); @@ -4803,6 +4819,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddExpressionStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5127,6 +5144,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddExpressionStored: /* ALTER COLUMN ADD GENERATED ALWAYS + * STORED USING CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddExpressionStored(rel, cmd, recurse, recursing, lockmode); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5521,6 +5546,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddExpressionStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddExpressionStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6403,13 +6434,23 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); +#endif + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); +#ifdef USE_INJECTION_POINTS + INJECTION_POINT("alter-table-phase-3-verify", NULL); +#endif + } if (newrel) { @@ -6729,6 +6770,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddExpressionStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8876,6 +8919,442 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddExpressionStored(Relation rel, + AlterTableCmd *cmd, + bool recurse, bool recursing, + LOCKMODE lockmode) +{ + /* + * Reject ONLY if there are child tables. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), lockmode)) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too"))); + + /* + * Cannot change only inherited columns to be stored generated columns. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errmsg("cannot change inherited column to be a stored generated column"))); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddExprStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData keys[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&keys[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&keys[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&keys[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, keys); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a serial column to a stored generated column"), + errdetail("\"%s\" of relation \"%s\" depends on sequence %s", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column referenced in a default expression to a stored generated column"), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + default: + /* We're not interested in the row */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, NoLock); +} + +/* + * Subroutine for ATExecAddExpressionStored, used to find a CHECK constraint + * to prove that the column values statisfy what will be the generator + * expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape. + * + * If the column is nullable: + * CHECK (column IS NOT DISTINCT FROM expr) + * + * If the column is NOT NULL, any of: + * CHECK (column IS NOT DISTINCT FROM expr) + * CHECK (column = expr) + * + * If a valid constraint is found, this returns both the Oid of the constraint + * and the unpacked expression. + */ +static Node * +findUsableConstraintForAddExprStored(Relation rel, AttrNumber attnum, + bool attisnotnull, + const char *conname) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key, + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 1, &key); + + foundExpr = NULL; + + while (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + char *conbin; + Datum val; + Node *conexpr; + + if (con->contype != CONSTRAINT_CHECK) + continue; + if (strcmp(conname, NameStr(con->conname)) != 0) + continue; + /* !conenforced implies !convalidated, but let's be explicit about it */ + if (!con->convalidated || !con->conenforced) + continue; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + /* Try to match IS NOT DISTINCT */ + if (IsA(conexpr, BoolExpr)) + { + BoolExpr *negation = (BoolExpr *) conexpr; + + if (list_length(negation->args) == 1 + && negation->boolop == NOT_EXPR + && IsA(linitial(negation->args), DistinctExpr)) + { + DistinctExpr *dist = linitial(negation->args); + + Assert(list_length(dist->args) == 2); + + if (IsA(linitial(dist->args), Var)) + { + Var *var = linitial(dist->args); + + if (var->varattno == attnum && + op_mergejoinable(dist->opno, exprType((Node *) var))) + { + foundExpr = lsecond(dist->args); + break; + } + } + } + } + /* If the column is NOT NULL, try to match = as well */ + if (attisnotnull && IsA(conexpr, OpExpr)) + { + OpExpr *op = (OpExpr *) conexpr; + + if (list_length(op->args) == 2 && IsA(linitial(op->args), Var)) + { + Var *var = linitial(op->args); + + if (var->varattno == attnum && + op_mergejoinable(op->opno, exprType((Node *) var))) + { + foundExpr = lsecond(op->args); + break; + } + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED ALWAYS STORED USING CONSTRAINT name + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddExprStored). + */ +static ObjectAddress +ATExecAddExpressionStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("Cannot convert an identity column to a stored generated column"), + errdetail("column \"%s\" of relation \"%s\" is an identity column", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("column \"%s\" of relation \"%s\" is already a generated column", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column if it's referenced by a partition key"), + errdetail("column \"%s\" is part of the partition key of relation \"%s\"", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddExprStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddExprStored(rel, attnum, + attTup->attnotnull, + def->conname); + if (foundConstraintExpr == NULL) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert a column into a stored generated column without a constraint to prove that the values are consistent"), + attTup->attnotnull ? + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" = expr) or CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName, + colName) : + errdetail("could not find a valid constraint \"%s\" CHECK (\"%s\" IS NOT DISTINCT FROM (expr))", + def->conname, + colName))); + + /* Mark as generated stored in pg_attribute */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + attTup->attgenerated = ATTRIBUTE_GENERATED_STORED; + CatalogTupleUpdate(pg_attribute, &tuple->t_self, tuple); + table_close(pg_attribute, RowExclusiveLock); + + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index ff4e1388c55..7bbde3d4e23 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2725,6 +2725,36 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED ALWAYS STORED USING CONSTRAINT constraint_name */ + | ALTER opt_column ColId ADD_P GENERATED generated_when STORED USING CONSTRAINT name + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $10; + c->contype = CONSTR_GENERATED; + c->generated_when = $6; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @10; + + /* + * Like in the case of ColConstraintElem, we cannot handle + * this in the grammar because IDENTITY allows both ALWAYS + * and BY DEFAULT, while generated columns only allow + * ALWAYS. This would lead to shift/reduce conflicts. + */ + if (c->generated_when != ATTRIBUTE_IDENTITY_ALWAYS) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("for a generated column, GENERATED ALWAYS must be specified"), + parser_errposition(@6))); + + n->subtype = AT_AddExpressionStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 64e27ef87a3..5d381433004 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -46,6 +46,8 @@ $node->safe_psql('postgres', . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" . "CREATE PUBLICATION some_publication;\n" . "CREATE TABLE fpo_test (id int4range, valid_at daterange, name text);\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (b IS NOT DISTINCT FROM (a + 1));\n" ); # In a VPATH build, we'll be started in the source directory, but we want @@ -460,6 +462,23 @@ check_completion("FR\t", qr/FROM /, clear_query(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("A\t", qr/ALWAYS /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS"); + +check_completion("S\t", qr/STORED USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT"); + +check_completion("\t\t", qr/check_gen/, + "complete ALTER COLUMN <col> ADD GENERATED ALWAYS STORED USING CONSTRAINT offers check constraint names"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index e4bc2c93145..4c6f05f8056 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -1884,6 +1893,7 @@ psql_completion(const char *text, int start, int end) #define prev7_wd (previous_words[6]) #define prev8_wd (previous_words[7]) #define prev9_wd (previous_words[8]) +#define prev10_wd (previous_words[9]) /* Match the last N words before point, case-insensitively. */ #define TailMatches(...) \ @@ -2970,12 +2980,34 @@ match_previous_words(int pattern_id, else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY", "STORED USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev10_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS STORED USING + * CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS", "STORED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev9_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index e03556399ab..7906bb4f9eb 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2525,6 +2525,8 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddExpressionStored, /* add generated always stored using + * constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index c01d2fb095c..ad214a8543e 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -9,7 +9,7 @@ EXTENSION = injection_points DATA = injection_points--1.0.sql PGFILEDESC = "injection_points - facility for injection points" -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress ISOLATION = basic \ diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..27dcc0e17a7 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 59dba1cb023..bd431c99cd1 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -36,6 +36,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..b9599bec175 --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (b = a * 2); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED ALWAYS STORED USING CONSTRAINT c2; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 64a1dfa9f79..fb5ab479aab 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddExpressionStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index e167a41ce79..075a2dd7bb6 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4876,3 +4876,479 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +insert into testgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +insert into testgen.t1 (a, b) values (10, 21); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +drop table testgen.t1; +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 + Table "testgen.t1" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause_equal" CHECK (b = (a * 2)) +Not-null constraints: + "t1_b_not_null" NOT NULL "b" + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; + a | b | expected | correct +----+----+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t + 6 | 12 | 12 | t + 7 | 14 | 14 | t + 8 | 16 | 16 | t + 9 | 18 | 18 | t + 10 | 20 | 20 | t +(10 rows) + +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause_does_not_exist" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" IS NOT DISTINCT FROM (expr)) +drop table testgen.t1; +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (b = (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("b" = expr) or CHECK ("b" IS NOT DISTINCT FROM (expr)) +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; + did_rewrite +------------- + f +(1 row) + +\d+ testgen.t4 + Table "testgen.t4" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+---------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | not null | | plain | | +Check constraints: + "chk_gen_clause" CHECK (b >= (a * 2)) +Not-null constraints: + "t4_b_not_null" NOT NULL "b" + +drop table testgen.t4; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from testgen.t5; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 + Table "testgen.t5" + Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +--------+---------+-----------+----------+------------------------------------+---------+--------------+------------- + a | integer | | | | plain | | + b | integer | | | generated always as (a * 2) stored | plain | | +Check constraints: + "chk_gen_clause" CHECK (NOT b IS DISTINCT FROM (a * 2)) + +select * from testgen.t5 order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +drop table testgen.t5; +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; + a | b | expected | correct +---+---+----------+--------- + 1 | 2 | 2 | t + 2 | 4 | 4 | t +(2 rows) + +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; + a | b | expected | correct +---+----+----------+--------- + 3 | 6 | 6 | t + 4 | 8 | 8 | t + 5 | 10 | 10 | t +(3 rows) + +rollback; +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +drop table testgen.tpart; +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); +ERROR: relation "testgen.tpart" does not exist +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column into a stored generated column without a constraint to prove that the values are consistent +DETAIL: could not find a valid constraint "chk_gen_clause" CHECK ("c" IS NOT DISTINCT FROM (expr)) +rollback; +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: ALTER COLUMN / ADD GENERATED ALWAYS STORED must be applied to child tables too +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +ERROR: cannot change inherited column to be a stored generated column +rollback; +drop table testgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table testgen.intermediate +drop cascades to table testgen.leaf +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; +ERROR: relation "doesnotexist" does not exist +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); +ERROR: argument of CHECK must be type boolean, not type integer +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; +ERROR: column "doesnotexist" of relation "t1" does not exist +alter table testgen.t1 add column b int; +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; +ERROR: for a generated column, GENERATED ALWAYS must be specified +LINE 2: add generated by default stored using constraint chk_gen... + ^ +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +ERROR: syntax error at or near ";" +LINE 1: alter table testgen.t1 alter column b add generated always; + ^ +alter table testgen.t1 alter column b add generated always virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...able testgen.t1 alter column b add generated always virtual; + ^ +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +ERROR: syntax error at or near "using" +LINE 1: ...le testgen.t1 alter column b add generated always using cons... + ^ +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +ERROR: syntax error at or near "virtual" +LINE 1: ...le testgen.t1 alter column b add generated always virtual us... + ^ +drop table testgen.t1; +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: column "b" of relation "t2" is already a generated column +drop table testgen.t2; +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +ERROR: Cannot convert an identity column to a stored generated column +DETAIL: column "b" of relation "t2" is an identity column +drop table testgen.t2; +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a serial column to a stored generated column +DETAIL: "b" of relation "t2" depends on sequence sequence testgen.t2_b_seq +drop table testgen.t2; +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot convert a column referenced in a default expression to a stored generated column +DETAIL: Column "c" is referenced by generated column "b". +drop table testgen.t3; +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table testgen.t3; +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +ERROR: cannot convert a column into a stored generated column if it's referenced by a partition key +DETAIL: column "c" is part of the partition key of relation "t3" +drop table testgen.t3; +set search_path to :search_path; +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +ERROR: generation expression is not immutable +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; +drop schema testgen cascade; diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index f5f13bbd3e7..04e4c6e619d 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3159,3 +3159,302 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED ALWAYS STORED USING CONSTRAINT name +-- turning a regular column into a stored generated column without a rewrite +create schema testgen; + +create table testgen.t1 (a int, b int); +insert into testgen.t1 (a, b) + select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct + from testgen.t1 order by a; +insert into testgen.t1 (a, b) values (10, 20); +insert into testgen.t1 (a, b) values (10, 21); +drop table testgen.t1; + +-- accepts = instead of IS NOT DISTINCT FROM when the destination +-- column is NOT NULL +create table testgen.t1 (a int, b int not null); +insert into testgen.t1 (a, b) +select x, x * 2 from generate_series(1, 10) x; +alter table testgen.t1 add constraint chk_gen_clause_equal check (b = a * 2); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_equal; +\d+ testgen.t1 +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.t1 order by a; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist +create table testgen.t1 (a int, b int); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint does not exist. When the destination +-- column is NOT NULL, the error message mentions both constraint +-- shapes which would be valid +create table testgen.t1 (a int, b int not null); +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause_does_not_exist; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not valid +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- fails when the constraint is not enforced +create table testgen.t1 (a int, b int); +alter table testgen.t1 add constraint chk_gen_clause check (b is not distinct from a * 2) not enforced; +alter table testgen.t1 alter column b + add generated always stored using constraint chk_gen_clause; +drop table testgen.t1; + +-- turning a regular column into a stored generated column +-- without rewriting the table doesn't touch the index either +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b = a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b + add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before = :t4_filenode_after as did_skip_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- turning a regular column into a stored generated column +-- fails when the constraint exists but doesn't have the expected shape +create table testgen.t4 (a int, b int not null); +insert into testgen.t4 (a, b) select x, x * 2 from generate_series(0, 5) x; +alter table testgen.t4 add constraint chk_gen_clause check (b >= a * 2); +select pg_relation_filenode('testgen.t4') as t4_filenode_before \gset +alter table testgen.t4 alter column b add generated always stored using constraint chk_gen_clause; +select pg_relation_filenode('testgen.t4') as t4_filenode_after \gset +select :t4_filenode_before != :t4_filenode_after as did_rewrite; +\d+ testgen.t4 +drop table testgen.t4; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks +create table testgen.t5 (a int); +select pg_relation_filenode('testgen.t5') as t5_filenode_before \gset +insert into testgen.t5 select x from generate_series(1, 5) x; +-- test nulls, too +insert into testgen.t5 (a) values (null); +alter table testgen.t5 add column b int; +-- take care of new and updated columns +create function testgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger testgen_gen + before insert or update on testgen.t5 + for each row execute function testgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table testgen.t5 + add constraint chk_gen_clause check (b is not distinct from a * 2) not valid; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +insert into testgen.t5 (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update testgen.t5 set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table testgen.t5 validate constraint chk_gen_clause; +select locktype, mode from pg_locks + where relation = 'testgen.t5'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table testgen.t5 alter column b + add generated always stored using constraint chk_gen_clause; +select locktype, mode from pg_locks +where relation = 'testgen.t5'::regclass and granted; +commit; +select pg_relation_filenode('testgen.t5') as t5_filenode_after \gset +select :t5_filenode_before = :t5_filenode_after as did_skip_rewrite; +select * from testgen.t5; +-- verify that it's still possible to insert rows (the trigger is still +-- running at this point) +insert into testgen.t5 (a) values (400); +drop trigger testgen_gen on testgen.t5; +drop function testgen.gen(); +insert into testgen.t5 (a) values (500); +\d+ testgen.t5 +select * from testgen.t5 order by a nulls first; +drop table testgen.t5; + +-- test support for partitioned tables and inheritance +create table testgen.tpart (a int, b int) partition by hash (a); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a * 2); +create table testgen.tpart_p1 partition of testgen.tpart + for values with (modulus 2, remainder 0); +create table testgen.tpart_p2 partition of testgen.tpart + for values with (modulus 2, remainder 1); +insert into testgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering the parent table, recursing +begin; +alter table testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +-- expected: all the partitions have been rewritten +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p1 order by a; +select a, b, a * 2 as expected, b = (a * 2) as correct +from testgen.tpart_p2 order by a; +rollback; + +-- altering a single partition is not allowed +begin; +-- expected: error +alter table testgen.tpart_p1 alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +-- altering only the parent table is not allowed +begin; +-- expected: error +alter table only testgen.tpart alter column b + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.tpart; + +-- test support for inheritance and subpartitions +create table testgen.root (a int, b int, c int); +create table testgen.intermediate () inherits (testgen.root); +create table testgen.leaf () inherits (testgen.intermediate); +alter table testgen.tpart + add constraint chk_gen_clause check (b is not distinct from a + b); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +-- ... hence all these should result in an error +begin; +alter table only testgen.root alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.intermediate alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; +begin; +alter table only testgen.leaf alter column c + add generated always stored using constraint chk_gen_clause; +rollback; + +drop table testgen.root cascade; + +-- tests for invalid invocations +alter table doesnotexist alter column foo + add generated always stored using constraint cdoesnotexist; + +create table testgen.t1 (a int); +alter table testgen.t1 add constraint chk_gen_clause check (1); + +alter table testgen.t1 alter column doesnotexist + add generated always stored using constraint chk_gen_clause; + +alter table testgen.t1 add column b int; + +-- invalid: only supports ALWAYS +alter table testgen.t1 alter column b + add generated by default stored using constraint chk_gen_clause; + +-- invalid: only supports STORED. These are all syntax errors. +alter table testgen.t1 alter column b add generated always; +alter table testgen.t1 alter column b add generated always virtual; +alter table testgen.t1 alter column b add generated always using constraint chk_gen_clause; +alter table testgen.t1 alter column b add generated always virtual using constraint chk_gen_clause; +drop table testgen.t1; + +-- invalid: b is already a generated column +create table testgen.t2 (a int, b int generated always as (a * 2) stored); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is an identity column +create table testgen.t2 (a int, b int generated always as identity); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; +create table testgen.t2 (a int, b int generated by default as identity ); +alter table testgen.t2 alter column b add generated always stored using constraint doesnotexist; +drop table testgen.t2; + +-- invalid: b is a serial column +create table testgen.t2 (a int, b bigserial); +alter table testgen.t2 add constraint chk_gen_clause check (b is not distinct from (1)); +alter table testgen.t2 alter column b add generated always stored using constraint chk_gen_clause; +drop table testgen.t2; + +-- invalid: c is referenced by another column's default expr +create table testgen.t3 (a int, b int generated always as (c + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c references another generated column +create table testgen.t3 (a int, b int generated always as (a + 1), c int); +alter table testgen.t3 add constraint chk_gen_clause check (c is not distinct from (b + 1)); +alter table testgen.t3 alter column c add generated always stored using constraint chk_gen_clause; +drop table testgen.t3; + +-- invalid: c is referenced in a partition key +create table testgen.t3 (a int, b int, c int) partition by hash (c); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table testgen.t3 (a int, b int, c int) partition by hash ((c)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +show search_path \gset +set search_path to testgen, public; +create table t3 (a int, b int, c int) partition by range ((t3)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +create table t3 (a int, b int, c int) partition by range ((t3 is null)); +alter table testgen.t3 alter column c add generated always stored using constraint doesnotexist; +drop table testgen.t3; +set search_path to :search_path; + +create table testgen.t3 (a int, b int); +-- invalid: expr must be immutable +alter table testgen.t3 add constraint chk_gen_clause check (b is not distinct from (a + random()::int)); +alter table testgen.t3 alter column b + add generated always stored using constraint chk_gen_clause; +alter table testgen.t3 drop constraint chk_gen_clause; +drop table testgen.t3; + +drop schema testgen cascade; base-commit: 6d4ca6de97770cdaee18517dd2f8fe8f4ecee187 -- 2.47.0 --vszqhseuea6ckzmb-- ^ permalink raw reply [nested|flat] 276+ messages in thread
end of thread, other threads:[~2026-07-03 05:52 UTC | newest] Thread overview: 276+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2025-08-11 14:12 [PATCH 4/4] Add CONCURRENTLY option to REPACK command. Antonin Houska <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-16 23:25 [PATCH v1 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-03-29 19:45 [PATCH v2 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-04-24 08:44 [PATCH v3 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-05-14 21:51 [PATCH v4 1/2] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-06-30 13:21 [PATCH v5] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]> 2026-07-03 05:52 [PATCH v6] Support changing a column into a stored generated column Alberto Piai <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox